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EXECUTIVE  SUMMARY 


1.0  Introduction 

This  summary  serves  as  a  stand-alone  document,  as  well  as  part  of  this  report.  Therefore,  there  is 
some  duplication  of  text  as  well  as  tables  and  figures  between  this  summary  and  the  full  report. 

2.0  JADS  Overview 

The  Joint  Advanced  Distributed  Simulation  (JADS)  Joint  Test  and  Evaluation  (JT&E)  was 
chartered  by  the  Deputy  Director,  Test,  Systems  Engineering  and  Evaluation  (Test  and 
Evaluation),  Office  of  the  Secretary  of  Defense  (Acquisition  and  Technology)  in  October  1994  to 
investigate  the  utility  of  advanced  distributed  simulation  (ADS)  technologies  for  support  of 
developmental  test  and  evaluation  (DT&E)  and  operational  test  and  evaluation  (OT&E).  The 
JADS  Joint  Test  Force  (JTF)  is  Air  Force  led  with  Army  and  Navy  participation.  The  JADS 
JT&E  program  is  currently  scheduled  to  end  in  March  2000. 

The  JADS  JTF  is  investigating  ADS  applications  in  three  slices  of  the  test  and  evaluation  (T&E) 
spectrum:  ADS  support  of  air-to-air  missile  testing;  ADS  support  for  command,  control, 
communications,  computers,  intelligence,  surveillance,  and  reconnaissance  (C4ISR)  testing;  and 
the  Electronic  Warfare  (EW)  Test  which  is  exploring  ADS  support  for  EW  testing. 

3.0  EW  Test  Overview 

The  tasking  to  conduct  an  ADS-based  EW  test  called  for  an  airborne  self-protection  jammer  as 
the  system  under  test  (SUT).  In  the  summer  of  1995,  JADS  presented  a  comprehensive  test  and 
analysis  approach  for  an  EW  test  to  the  JT&E  technical  advisory  board  (TAB)  and  the  senior 
advisory  council  (SAC).  The  JADS  EW  Test  approach  was  fully  supported  by  the  TAB  but  not 
chartered  initially  due  primarily  to  the  high  cost  ($18  million).  In  response,  JADS  tailored  the 
initial  EW  Test  design  and  subsequently  developed  a  reduced  scope,  lower  cost  test  and  analysis 
approach  using  the  ALQ-131  jammer  pod  as  the  SUT. 

The  emphasis  of  the  EW  Test  was  on  the  performance  of  the  ADS  components  and  their 
contribution  or  impact  to  testing  rather  than  on  the  performance  of  the  ALQ-131  pod  itself. 
Measures  of  performance  (MOPs)  for  the  jammer  were  identified  as  measures  that  would  most 
likely  be  affected  by  distributed  testing.  Computing  and  correlating  the  MOPs  became  a 
methodology  for  evaluating  ADS.  JADS  will  evaluate  distributed  test  control  and  analysis, 
network  performance,  relationships  between  data  latencies,  and  ADS-induced  data  anomalies. 
Time,  cost,  and  complexity,  as  well  as  validity  and  credibility  of  the  data,  are  part  of  the 
evaluation. 
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The  EW  Test  was  designed  as  a  three-phase  effort  providing  a  baseline  of  jammer  performance 
data  in  a  non-ADS  environment  which  will  then  be  compared  to  multiple  tests  of  the  same 
configuration  in  an  ADS  environment. 

Phase  1  (the  subject  of  this  report)  included  (1)  a  risk  reduction  flight  test  effort  at  the  Western 
Test  Range  (WTR),  (2)  a  baseline  flight  test  using  an  ALQ-131  jamming  pod  at  the  WTR,  (3)  a 
short  hardware-in-the-loop  (HITL)  test  at  the  Air  Force  Electronic  Warfare  Environment 
Simulator  (AFEWES)  at  Fort  Worth,  Texas,  and  (4)  a  system  integration  laboratory  (SIL)  test  at 
the  Automatic  Multiple  Environment  Simulator  (AMES)  facility  at  Eglin  Air  Force  Base  (AFB), 
Florida.  The  HITL  and  SIL  tests  were  added  to  supplement  the  baseline  flight  testing  and  to 
provide  missing  data.  This  established  a  baseline  of  environment  and  jammer  performance  data 
against  two  command-guided  surface-to-air  missile  (SAM)  sites,  one  semi-active  surface-to-air 
missile  site,  and  one  anti-aircraft  artillery  (AAA)  site.  This  scenario  was  used  to  develop  the  ADS 
test  environment  for  the  two  subsequent  ADS  test  phases  and  provided  the  baseline  data  for 
comparison  with  the  ADS  test  results.  Additionally,  the  performance  data  provided  a  baseline  for 
attempting  correlation  ability  across  all  three  phases  of  the  test. 

Phase  2  will  be  a  test  of  a  real-time  digital  system  model  (DSM)  of  the  ALQ-131  receiver 
processor  linked  with  terminal  threats  at  the  AFEWES  facility  and  a  scripted  model  of  the 
terminal  threat  hand-off  portion  of  an  integrated  air  defense  system  (IADS).  The  threat  laydown 
used  in  the  Phase  1  flights  will  be  replicated  in  the  synthetic  ADS  environment;  the  jammer  model 
will  be  flown,  via  the  scripted  flight  profiles  developed  from  the  actual  open  air  range  (OAR) 
baseline  flights  and  HITL  test,  against  the  AFEWES  threats. 

Phase  3  will  be  a  test  using  the  ALQ-131  jammer  installed  on  an  F- 16  aircraft  located  in  the  Air 
Combat  Environment  Test  and  Evaluation  Facility  (ACETEF)  located  at  Patuxent  River  Naval 
Air  Station,  Maryland.  This  facility  will  be  linked  with  AFEWES  threats  using  the  same  threat 
laydown  as  the  previous  tests  and  controlled  by  the  same  scripted  flight  profile. 

4.0  Overview  of  EW  Test  Phase  1 
4.1  Purpose 

The  primary  purpose  of  this  phase  was  to  provide  baseline  data  for  verification  and  validation 
(V&V)  of  the  results  obtained  in  later  ADS  test  phases.  The  risk  reduction  test  missions  were 
conducted  from  28  August  1997  to  20  May  1998.  The  HITL  portion  was  conducted  at  the 
AFEWES  facility  from  27  July  to  7  August  1998.  OAR  baseline  flight  testing  was  conducted  at 
the  Western  Test  Range  from  8  June  to  20  August  1998.  The  SIL  testing  was  conducted  at  the 
AMES  facility  from  15  to  18  September  1998. 

Test  objectives  of  the  risk  reduction,  OAR  baseline,  HITL,  and  SIL  portions  of  the  EW  Test 
Phase  1  are  summarized  in  Table  ES-1 . 


2 


Table  ES-1.  EW  Phase  1  Objectives 


Objective  # 

Risk  Reduction  Objective 

1.1 

Verify  OAR  test  support  concept  is  feasible 

1.2 

Implement/evaluate  the  rules  of  engagement 

Verify  sufficient  information/experience  has  been  obtained  to  prepare  Phase  1 
documentation 

Collect  and  evaluate  Firefly  data 

2.2 

Collect  and  evaluate  ALQ-131  pod  data 

2.3 

Verify  planned  sorties  will  produce  sufficient  sample  size 

2.4 

Demonstrate  ability  to  collect  repeatable  data  on  the  OAR 

2.5 

Verify  OAR  data  can  be  replicated  in  an  ADS  environment 

3.1 

Gather  engineering  data  to  verify  data  analysis  tools 

3.2 

Exercise  data  analysis  path/process 

3.3 

Verify  analysis  techniques,  tools,  and  products 

4.1 

Evaluate  observer  training  effectiveness 

4.2 

Evaluate  analyst  training  effectiveness 

5.1 

Complete  individual  mission  quick-look  reports 

5.2 

Complete  final  risk  reduction  phase  report 

OAR  Baseline  Objective 

l.i 

Establish  jammer  performance  in  an  OAR  environment 

1.2 

Establish  the  repeatability  of  OAR  baseline  test  results 

1.3 

Establish  ranges  of  OAR  baseline  statistics  for  event  data 

1.4 

Establish  range  of  correlation  coefficients  for  series  observables 

Ob  jective  # 

HITL  Objective 

1.1 

Establish  jammer  performance  in  a  HITL  environment 

1.2 

Establish  the  repeatability  of  HITL  test  results 

1.3 

Establish  ranges  of  HITL  statistics  for  event  data 

1.4 

Establish  range  of  correlation  coefficients  for  series  observables 

SIL  Objective 

l.i 

Collect  60  samples  of  jammer  internal  timing  for  each  jammer  change  of  state 

4.2  Approach 

The  risk  reduction  and  OAR  baseline  missions  were  flown  by  an  Air  National  Guard  Air  Reserve 
Test  Center  (AATC)  F-16  test  aircraft  with  the  ALQ-131  jammer  and  airborne  instrumentation 
over  the  Western  Test  Range.  The  ALQ-131  was  employed  against  a  limited  ground  threat 
environment  using  a  custom  software  tape  with  specific  receive  and  transmit  functions  tailored  to 
the  four  threats  in  the  JADS  EW  Test  scenario.  The  custom  tape  also  implemented  ECM 
technique  changes  that  altered  the  effectiveness  of  the  jammer.  These  changes,  while  reducing 
ALQ-131  effectiveness,  were  made  to  assist  JADS  in  better  evaluating  the  impacts  of  ADS. 
These  modifications  limited  threat  scenario  variability  and  maximized  repeatability  aiding 
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correlation  assessment  during  later  test  phases.  A  stabilized  test  scenario  and  threat  operator 
procedures  evolved  during  risk  reduction  and  were  employed  throughout  the  open  air  testing. 

HITL  testing  employed  one  of  the  ALQ-131  pods  flown  during  the  OAR  baseline  testing  as  the 
“hardware”  in  the  hardware-in-the  loop  during  the  test  at  AFEWES.  Sixty-two  OAR  baseline 
mission  runs  were  scripted  and  executed  at  AFEWES,  replicating  the  OAR  baseline  scenario  and 
rules  of  engagement  (ROE)  as  closely  as  possible.  The  HITL  data  were  subsequently  compared 
with  OAR  data  for  correlation  purposes. 

The  SIL  test  at  Eglin  AFB,  Florida,  was  created  and  executed  in  three  weeks.  The  rules  of 
engagement  and  the  geometry  of  the  OAR  baseline  were  used  to  develop  a  set  of  computer 
instructions  that  activated  the  emitters  at  the  proper  time  in  the  flight  profile  and  conditioned  the 
radio  frequency  (RF)  signals  to  represent  the  correct  signal  strength  to  the  pod.  In  this  case,  we 
were  only  interested  in  internal  jammer  timing  MOPs,  and  all  four  threats  were  active  and 
controlled  by  the  computer. 

5.0  Phase  1  Test  Results 

5.1  Scheduling 

The  original  risk  reduction  schedule  called  for  six  missions  to  be  flown  between  August  and 
November  1997.  Flight  testing  did  begin  on  27  August  1997  with  the  first  mission;  however, 
technical  problems  with  the  ALQ-131  operational  flight  program  (OFP),  instrumentation,  and 
electronic  countermeasures  (ECM)  technique  response  software  programs  delayed  the  second 
mission  until  18  November  1997.  Similar  problems  combined  with  test  range  and  AATC 
scheduling  conflicts,  asset  nonavailability,  and  data  delivery  problems  continued  to  cause  delays  in 
mission  scheduling  throughout  the  risk  reduction  phase.  The  sixth  and  final  risk  reduction  mission 
was  flown  on  20  May  1998. 

The  plan  was  to  fly  fourteen  productive  range  hours  to  complete  the  OAR  baseline  test  following 
the  six  risk  reduction  missions.  The  first  dedicated  OAR  baseline  mission  was  flown  on  8  June 
1998.  Some  of  the  same  scheduling/execution  problems  experienced  in  risk  reduction  continued 
to  plague  the  OAR  baseline  planning,  and  some  missions  flown  were  marginally  productive.  All 
of  these  problems  together  resulted  in  a  total  of  17  OAR  baseline  missions  of  1.5  to  2.0  hours  of 
range  time  being  scheduled  and  seven  missions  actually  being  flown.  The  total  number  of  range 
hours  was  14.4.  The  final  OAR  baseline  mission  was  flown  on  20  August  1998. 

The  HITL  test  was  not  part  of  the  Phase  1  test  baseline.  It  was  added  when  we  realized  we 
would  be  unable  to  collect  all  MOP  data  on  the  OAR.  The  HITL  portion  of  the  Phase  1  testing 
was  executed  on  schedule  during  the  last  week  of  July  and  the  first  week  of  August  1998.  Time 
synchronization  problems  at  AFEWES  precluded  accomplishment  of  several  timing  MOPs,  but 
the  other  objectives  were  accomplished.  Final  results  analysis  by  Georgia  Tech  Research  Institute 
(GTRI)  will  be  provided  under  separate  cover. 
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5.2  Fulfillment  of  Test  Objectives 


All  risk  reduction  and  OAR  baseline  test  objectives  were  met.  Even  though  the  HITL  test  results 
did  not  produce  answers  to  the  timing  MOPs,  the  Air  Warfare  Center  SIL  testing  did  accomplish 
this.  Thus  all  essential  Phase  1  objectives  were  essentially  completed.  OAR  baseline  mission 
success,  i.e.,  data  elements  completed  versus  attempted,  was  approximately  78  percent. 

6.0  Lessons  Learned 

The  major  lessons  learned  were  in  the  areas  of  instrumentation,  range  operations,  data  processing 
and  delivery,  and  support  agency  interface  as  shown  in  the  following  paragraphs. 

6.1  Instrumentation 

The  overall  lesson  learned  concerning  both  range  and  airborne  instrumentation  is  to  take  the  time 
prior  to  the  test  to  thoroughly  verity  the  real  capabilities  of  the  systems  proposed.  The  test 
planner  must  be  able  to  clearly  define  specific  requirements  to  the  test  agency  providing  the 
instrumentation  and  ask  the  pertinent  questions  to  find  out  whether  the  instrumentation  meets  the 
requirement.  In  most  cases,  modifications,  improvements,  and/or  procedural  changes  are  needed 
to  tailor  the  use  of  the  instrumentation  to  the  needs  of  the  individual  test.  Test  facilities  and 
instrumentation  suppliers  are  in  the  business  of  “selling”  the  capabilities  of  their  assets,  and  it  is  up 
to  the  test  manager  to  determine  their  true  capability/applicability  by  asking  the  hard  questions. 
We  were  constrained  to  existing  instrumentation  and  as  a  result  had  to  overcome  data  shortfalls. 
JADS  experience  with  the  Radar  Detection  and  Performance  Analysis  System  (RDAPAS)  assets 
and  the  ALQ-131  1553  databus  cards  are  vivid  examples  of  this  lesson. 

6.2  Range  Operations 
6.2.1  Mission  Control 

Test  control  was  a  source  of  execution  problems.  Premission  problems  were  reduced  by  using  a 
very  detailed  premission  checklist  and  by  the  EW  Test  personnel  deploying  to  the  range  prior  to 
each  mission  to  confirm  preparations  were  being  implemented  according  to  plan.  During  the 
mission,  the  mission  controller,  in  charge  of  calling  out  execution  condition  “marks,”  was 
sometimes  hampered  by  other  duties.  Besides  controlling  the  start  and  stop  of  each  mission,  the 
mission  controller  was  also  expected  to  handle  real-time  equipment  problems  as  they  occurred 
during  a  mission.  Because  of  this  task  loading,  execution  “marks”  were  called  late  many  times 
and  occasionally  missed.  This  situation  improved  late  in  the  OAR  baseline  flight  schedule  when 
the  other  duties  were  finally  delegated  to  someone  else. 

There  were  also  occasional  range  coordination  problems  such  as  the  incorrect  range  area  being 
requested  for  flight  clearance,  confusion  causing  video  recorders  not  to  be  turned  on,  execution 
scripts  not  available/not  reviewed  at  some  threat  sites,  and  the  test  aircraft  arriving  at  the  initial 
point  (IP)  too  early/late  for  the  start  of  the  range  time. 
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Test  controller  workload  should  be  analyzed  ahead  of  time  and  divided  among  two  or  more 
individuals  so  that  crucial  test  condition  calls  can  be  made  consistently  from  run  to  ran. 

6.2.2  Threat  Operating  Procedures 

The  initial  JADS  approach  allowed  the  threat  operators  to  use  their  best  judgment  in  employing 
the  threats  to  acquire  and  track  the  test  aircraft.  However,  this  approach  introduced  too  much 
variation  in  the  test  conditions.  Therefore,  a  site  controller  matrix  (see  Appendix  D)  was 
developed  which  reduced  threat  operation  variability  and  generated  more  consistent  run 
conditions.  However,  it  was  realized  that  even  with  the  carefully  written  down  site  controller 
matrix  and  the  ROE,  there  was  still  ambiguity.  The  cure  was  face-to-face  meetings  with  all 
operators  early  on.  This  meeting  was  followed  by  actual  over  the  shoulder  observation  and  pre- 
and  post-briefs  by  designated  site  observers. 

An  associated  problem  involved  the  threat  operators  following  the  rules  of  engagement 
consistently.  JADS  missions  were  sometimes  3-4  weeks  apart  and  the  operators  were  not  always 
the  same  from  mission  to  mission.  This  required  JADS  to  refresh  the  operators  on  JADS-peculiar 
operations  prior  to  each  mission  to  preclude  errors  in  following  the  ROE.  This  required  a  lot  of 
extra  time  for  JADS  observers  to  “retrain”  the  operators. 

A  detailed  threat  operations  matrix  will  help  ensure  consistent  threat  tum-on/off  times,  mode 
changes,  and  rules  of  engagement  procedures  for  a  structured  test  such  as  the  JADS  EW  Test. 
Also,  if  missions  are  separated  by  weeks,  the  operators  need  to  be  refreshed  on  operating 
procedures  prior  to  each  mission  to  ensure  enforcement  of  the  ROE.  This  may  not  be  desirable  in 
a  more  OT&E-oriented  test  but  is  necessary  when  precise  baseline  data  are  needed.  The 
communication  process  with  threat  operators  is  more  complicated  than  it  would  appear.  For 
example,  the  “no  electronic  counter-countermeasure  (ECCM)”  ROE  seemed  clear  and 
unambiguous.  However,  only  after  a  series  of  meetings  were  all  of  the  ambiguities  cleared  up. 

6.2.3  Communications 

Communication  among  JADS  personnel  in  the  mission  control  room  and  observers  at  individual 
sites  was  inadequate  at  first,  but  improved  markedly  with  the  establishment  of  a  dedicated 
observer  net  in  addition  to  the  customer  net.  These  nets  allowed  direct  feedback  and  real-time 
test  condition  adjustments  when  anomalies  were  noted. 

A  separate  test  observer  communications  link  between  the  mission  control  center  and  individual 
threat  sites  is  very  beneficial  to  detect  problems  and  address  them  in  near  real  time. 
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6.3  Data  Processing  and  Delivery 
6.3.1  Data  Processing  Tools 

Data  processing  was  supposed  to  be  enhanced  by  the  existence  of  an  automated  data  reduction 
application.  The  application  chosen  was  the  Automated  Data  Reduction  Software  (ADRS),  a 
GTR1  product  which  evolves  with  each  new  customer  and  the  needs  which  they  bring  to  the  table. 
After  a  series  of  discussions  with  the  WTR  analysts,  significant  progress  was  made  in  finalizing 
data  sources,  formats,  and  collection  rates.  However,  the  software  was  not  completely  ready 
before  the  beginning  of  testing  and  refinements  continued  beyond  Phase  1  execution. 

Our  first  ADRS  version  was  plagued  by  software  bugs  which  caused  the  system  to  crash 
frequently.  Through  diligent,  comprehensive  feedback  to  the  GTRI  programmer,  nearly  all  these 
bugs  were  fixed.  Even  though  GTRI  was  responsive,  these  fixes  took  place  midstream  and  they 
hampered  data  reduction  and  analysis. 

The  second  problem  involved  JADS  special  data  analysis  requirements  for  a  small  number  of  the 
MOPs.  More  ADRS  code  needed  to  be  built  to  handle  these  requirements.  This  recoding  took 
place  while  OAR  baseline  missions  were  being  flown.  This  overlap  impeded  reporting  final  results 
on  every  MOP. 

The  third  problem  resulted  from  changing  MOP  calculation  methods  after  receipt  of  the  first 
version  of  ADRS.  The  definition  and  calculation  of  four  MOPs  continued  to  evolve  throughout 
Phase  1  and  Phase  2  execution  to  allow  for  better  correlation  and  repeatability  in  the  test. 

Currently,  the  data  analysis  and  reduction  tools  used  for  the  OAR  baseline  missions  are 
cumbersome,  often  slow  to  execute,  and  some  utilities  do  not  work  (e.g.,  time  skip  function, 
MOP  data  export  function).  The  preparatory  tools  used  on  OAR  baseline  mission  data  are  the 
ALQ-131  digibus  reader,  EMV  converter,  Dbase  utility,  and  the  Firefly  split  tool.  All  of  these 
tools  use  separate  application  interfaces  that  are  often  clumsy  and  difficult  to  understand 
(especially  the  ALQ-131  digibus  reader).  This  makes  it  very  difficult  for  new  team  members  and 
outside  personnel  to  understand  the  data  analysis  process  and  hinders  productivity. 

Contractor  developed  data  analysis  tools  should  be  completely  developed  and  demonstrated  prior 
to  the  start  of  test.  This  should  include  hands-on  practice  (training)  reducing  and  analyzing 
representative  data  expected  from  the  test.  Analysis  of  data  from  risk  reduction  missions  will  help 
work  out  the  bugs  in  the  data  analysis  tool  and  allow  for  improvements  prior  to  working  on  data 
from  the  missions  for  record. 

Data  reduction  tools  need  to  have  a  common  interface  that  will  call  upon  the  individual  tools. 
Moreover,  they  should  be  selectable  in  the  order  used  so  the  user  knows  what  actions  need  to  be 
performed  and  in  what  order. 
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6.3.2  Data  Delivery 

Timely  data  delivery  is  paramount  to  test  success.  A  number  of  the  JADS  EW  Test  risk  reduction 
missions  were  flown  almost  back  to  back,  and  it  was  not  possible  to  obtain  and  analyze  the  results 
from  one  mission  before  flying  the  next.  Quick  data  receipt  and  analysis  can  be  crucial  to 
troubleshooting  and  mission  success,  especially  during  a  risk  reduction  process. 

The  timeline  for  flight  test  data  reduction  and  analysis  is  just  as  important  as  the  mission  schedule 
timeline.  Efficiency  is  lost  (i.e.,  successive  missions  flown  with  bad  assets)  if  data  cannot  be 
examined  between  missions.  Missions  (especially  during  risk  reduction)  must  be  scheduled  with 
adequate  time  between  to  analyze  data  from  the  previous  mission.  Another  lesson  is  that  all 
means  must  be  exercised  to  expedite  delivery  of  range  and  aircraft  data  products  to  the  analysts 
whenever  mission  schedules  are  tight. 

6.4  Support  Agency  Interface 

6.4.1  Integrated  Product  Team  (IPT) 

The  IPT  helped  bridge  gaps  in  expertise  and  with  programmatic  decisions.  The  concept  of  an  IPT 
is  a  good  one.  However,  disadvantages  arise  with  the  geographical  separation  of  the  members  of 
the  IPT.  The  JADS  EW  Test  IPT  consists  of  members  resident  from  coast  to  coast. 
Teleconferencing  and  electronic  mail  (E-mail)  are  powerful  tools  that  overcame  some  of  these 
communication  obstacles;  however,  the  most  progress  is  made  when  members  meet  face-to-face. 
This  is  due  to  the  highly  technical  nature  of  the  work,  the  concepts  involved,  and  the 
software/hardware  which  are  used  on  this  test. 

When  technical  problems  arise,  teleconferences  and/or  E-mail  can  help  solve  many  of  the  issues, 
but  in  many  cases  face-to-face  meetings  between  key  players  are  needed  to  resolve  items.  This 
allows  direct  give-and-take  questions  to  reduce  incorrect  interpretations  and  confusion  of  what 
was  said  versus  what  was  meant.  Such  meetings  must  have  the  following  elements: 

•  Attendees  limited  to  those  individuals  from  each  key  agency  who  can  and  will  make  decisions 
and  commit  required  support  to  the  test 

•  Specific  agenda  including  status  updates  from  previous  actions  items 

•  Specific  action  items  assigned  to  individuals  by  name  with  suspense  dates 

•  Periodic  review  and  formal  tracking  of  action  item  suspenses 

6.4.2  AATC 

Normally  operations  went  smoothly  in  this  area.  JADS  received  outstanding  support  from  AATC 
ECM  pod  personnel.  However,  occasionally  data  were  lost  during  some  of  the  test  missions 
because  of  errors  in  pod  control  switch  actions  in  flight.  This  appeared  to  be  because  of  pilot 
unfamiliarity  and/or  lack  of  premission  briefing  on  the  pod  functions  and  controls.  A  number  of 
different  pilots  flew  the  various  test  missions,  and  some  were  very  familiar  with  ALQ-131  pod 
operation  and  peculiarities,  while  others  were  not. 
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At  times  JADS  had  difficulty  effectively  scheduling  AATC  test  aircraft,  ECM  pods,  and  personnel 
to  support  available  test  range  periods.  Scarce  assets  and  other  AATC  mission  priorities  as  well 
as  late  changes  to  range  schedules  contributed  to  this  problem.  A  number  of  missions  were  lost 
and  only  a  few  were  able  to  be  rescheduled  and  flown  because  of  the  limited  flexibility  of  AATC 
and  test  range  schedules. 

The  aircrew  must  be  thoroughly  briefed  on  operation  and  functions  of  the  system  under  test.  This 
is  especially  crucial  when  test  instrumentation  is  interfaced  with  the  test  item  that  may  affect  its 
normal  operation.  Whenever  feasible  within  aircrew  scheduling  limitations,  the  same  aircrew 
members  should  fly  as  many  of  the  test  missions  as  possible  to  reduce  the  learning  curve  from 
mission  to  mission. 

The  lesson  learned  concerning  scheduling  problems  is  that  test  managers  must  get  firm  support 
agency  commitments  during  initial  test  planning  and  fully  understand  support  agency  and  range 
scheduling  flexibility  limitations  and  factor  those  into  the  overall  test  schedule. 

6.4.3  GTRI 

GTRI  technical  expertise  was  crucial  to  the  success  of  the  JADS  EW  Test  program.  GTRI  direct 
interaction  with  JADS  EW  Test  team  members  was  needed  to  address  a  number  of  data  analysis 
issues  during  risk  reduction.  Lack  of  effective  communication  among  individuals  during  this 
process  sometimes  was  a  problem  which  impacted  solving  issues.  The  technical  expertise 
required  was  resident  within  the  GTRI  staff;  however,  it  was  difficult  to  access  it  because  JADS 
requested  a  single  point  of  contact  (POC)  (GTRI  program  manager)  to  coordinate  the  contracted 
technical  expertise.  Because  of  this  bottleneck,  the  team  was  often  unable  to  directly  interface 
with  the  individual  GTRI  expert  needed  to  solve  a  particular  problem  in  a  timely  manner.  This 
also  unnecessarily  burdened  the  program  manager  with  personnel  management  actions  instead  of 
allowing  time  to  work  technical  problems  effectively.  Critical  details  were  lost  transmitting 
information  through  multiple  nodes  in  the  organization  and  progress  was  impeded.  This  led  to 
many  miscommunications,  misunderstandings,  delays,  and  undue  stress  on  the  GTRI  POC  and 
JADS  team  members. 

Having  a  contractual  single  point  of  contact  to  coordinate  support  can  be  a  good  management 
technique;  however,  that  POC  must  be  allowed  to  be  flexible  enough  to  arrange  for  and  allow 
direct  working  relationships  among  technical  experts  and  the  test  team  members.  Such 
agreements  should  be  established  early  in  the  program  and  clearly  defined  to  the  satisfaction  of  all 
parties. 

7.0  Conclusions 

JADS  was  able  to  collect  an  adequate  baseline  data  set  for  correlation  with  the  ADS-based  test 
phases.  The  JADS  EW  Test  scenario  was  designed  to  reduce  the  run-to-run  variation.  The 
scenario  was  scripted  as  much  as  possible  to  limit  the  variation  within  each  facility  and  across  the 
facilities.  This  was  largely  successful.  However,  the  test  scenario  stressed  each  facility’s  ability  to 
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reduce  the  run-to-run  variation.  The  primary  source  of  variation  both  within  and  across  facilities 
was  human  operator  and  test  control.  The  OAR  baseline  had  more  run-to-run  variation  than 
AFEWES.  A  number  of  factors  contributed  to  this.  The  time  between  missions  was  a  critical 
factor.  Better  results  were  obtained  when  missions  were  flown  on  successive  days.  Another 
factor  was  the  number  of  different  operators  from  mission  to  mission  at  the  OAR.  AFEWES  used 
only  one  team  of  operators  for  each  threat.  Finally,  a  cultural  difference  between  the  facilities  was 
evident.  AFEWES  is  primarily  a  development  test  facility  while  the  OAR  specializes  in 
operational  effectiveness  testing.  Development  tests  emphasize  tight  control  and  repeatability, 
while  the  operational  tests  emphasize  operator  free  play  and  skill. 

Data  results  are  classified  and  are  contained  in  a  separate  report  to  be  published  by  GTRI.  GTRI 
was  tasked  to  reduce  and  analyze  all  Phase  1  data  and  report  the  results.  Their  report  will  discuss 
each  MOP  and  provide  the  statistical  results  (mean  and  variance).  The  JADS  EW  Test  team  will 
attempt  to  identify  sources  of  variance  where  possible.  In  addition,  GTRI  will  perform  the 
correlation  analysis  of  OAR  baseline  and  HITL  data.  The  classified  report  will  describe  the 
baseline  data  that  JADS  will  use  for  correlation  with  the  ADS-based  test  phases. 


10 


1.0  Introduction 


1.1  Overview 

The  Joint  Advanced  Distributed  Simulation  (JADS)  Joint  Test  and  Evaluation  (JT&E)  program  is 
an  Office  of  the  Secretary  of  Defense  (OSD)-sponsored  joint  service  effort  designed  to  determine 
how  well  advanced  distributed  simulation  (ADS)  can  support  test  and  evaluation  activities.  The 
Electronic  Warfare  (EW)  Test  is  one  of  three  tests  comprising  the  JADS  Joint  Test  Force  (JTF). 
It  was  chartered  separately  in  1996  to  test  the  utility  of  distributed  simulation  to  the  EW  test  and 
evaluation  (T&E)  community.  This  report  focuses  on  results  of  the  EW  Test  Phase  1  which 
consisted  of  six  risk  reduction  flight  test  missions,  a  hardware-in-the-loop  (HITL)  effort,  seven 
baseline  open  air  range  (OAR)  flight  test  missions,  and  one  system  integration  laboratory  (SIL) 
effort.  The  primary  purpose  of  this  phase  was  to  provide  baseline  data  for  verification  and 
validation  (V&V)  of  the  results  obtained  in  later  ADS  test  phases.  The  risk  reduction  test 
missions  were  conducted  from  28  August  1997  to  20  May  1998.  The  HITL  portion  was 
conducted  at  the  Air  Force  Electronic  Warfare  Evaluation  Simulator  (AFEWES)  facility,  Fort 
Worth,  Texas,  from  27  July  to  7  August  1998.  OAR  baseline  flight  testing  was  conducted  at  the 
Western  Test  Range  (WTR),  from  8  June  to  20  August  1998.  The  SIL  testing  was  conducted  at 
the  Automatic  Multiple  Environment  Simulator  (AMES)  facility  at  Eglin  Air  Force  Base  (AFB), 
Florida,  from  15  to  18  September  1998. 

Background  information  on  the  history  and  planning  for  the  EW  Test  in  general  and  this  Phase  1 
effort  specifically  is  contained  in  the  references  listed  below.  It  should  be  noted  that  the  original 
planning  only  discussed  the  OAR  baseline  efforts.  The  HITL  and  SIL  were  required  to  overcome 
instrumentation  limitations  and  to  better  isolate  ADS  effects  from  differences  between  the  HITL 
and  OAR  environments.  The  HITL  and  SIL  are  discussed  in  paragraphs  2.1.3  and  2.1.4. 

1.2  References 

Program  Level  Test  Activity  Plan  and  Data  Management  and  Analysis  Plan  (TAP/DMAP),  March 
1998. 

Electronic  Warfare  Phase  1  TAP/DMAP,  July  1998. 

EW  Hardware-in-the  Loop  (HITL)  TAP/DMAP,  September  1998. 

Electronic  Warfare  Test  Analysis  Plan  for  Assessment  (APA),  May  1996. 

EW  Jammer  Risk  Reduction  Test  Plan,  July  1997. 

1.3  Electronic  Warfare  Test 

The  tasking  to  conduct  an  ADS-based  test  of  an  EW  system  specifically  called  for  the  use  of  an 
airborne  self-protection  jammer  as  the  system  under  test.  In  the  summer  of  1995,  JADS 
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presented  a  comprehensive  test  and  analysis  approach  for  an  EW  test  to  the  JT&E  technical 
advisory  board  (TAB)  and  the  senior  advisory  council  (SAC).  The  JADS  EW  Test  approach  was 
fully  supported  by  the  TAB  but  not  chartered  initially  due  primarily  to  the  high  cost  ($18  million). 
In  response,  JADS  tailored  the  initial  EW  Test  design  and  subsequently  developed  a  reduced 
scope,  lower  cost  test  and  analysis  approach  using  the  ALQ-1 31  jammer  pod. 

The  emphasis  of  the  EW  Test  was  on  the  performance  of  the  ADS  components  and  their 
contribution  to  testing  rather  than  on  the  performance  of  the  ALQ-131  pod  itself.  Measures  of 
performance  (MOPs)  for  the  jammer  were  identified  as  measures  that  were  likely  to  be  affected  by 
distributed  testing.  Computing  and  correlating  the  MOPs  became  a  methodology  for  evaluating 
ADS.  These  measures  are  listed  in  Table  1.  JADS  will  evaluate  distributed  test  control  and 
analysis,  network  performance,  relationships  between  data  latencies,  and  ADS-induced  data 
anomalies.  Time,  cost,  and  complexity,  as  well  as  validity  and  credibility  of  the  data,  are  part  of 
the  evaluation.  Specific  test  scenarios  were  selected  to  allow  this  comparison.  Additionally,  some 
test  activities  were  planned  that  would  not  be  feasible  without  ADS  technology. 

Table  1.  EW  Test  Measures  of  Performance 


MOP# 

Description 

1 

Correct  threat  identification  ! 

2~ 

Correct  threat  identification  response  time 

3 

Correct  electronic  counter  measures  (ECM)  technique 
selection 

4 

Correct  ECM  technique  selection  response  time 

5 

Jamming-to-signal  ratio  versus  threats 

6 

Increase  in  tracking  error  versus  threat 

7 

Number  of  breaklocks  ! 

8 

Reduction  in  engagement  time 

9 

Reduction  in  missiles  launched 

Missile  miss  distance 

1.3.1  EW  Test  Approach 

The  EW  Test  was  designed  as  a  three-phase  effort  providing  a  baseline  of  jammer  performance 
data  in  a  non-ADS  environment  which  will  then  be  compared  to  multiple  tests  of  the  same 
configuration  in  an  ADS  environment. 

Phase  1  included  (1)  an  open  air  range  (OAR)  risk  reduction  flight  test  effort  at  the  Western  Test 
Range  (WTR),  (2)  a  baseline  flight  test  using  an  ALQ-131  jamming  pod,  (3)  a  short  HITL  test  at 
the  AFEWES,  and  (4)  a  SIL  test  at  the  AMES  facility.  The  HITL  and  SIL  tests  were  added  to 
supplement  the  baseline  flight  testing  and  provide  missing  data.  The  purpose  of  this  test  phase 
was  to  establish  a  baseline  of  environment  and  jammer  performance  data  against  two  command- 
guided  surface-to-air  missile  (SAM)  sites,  one  semi-active  surface-to-air  missile  site,  and  one  anti¬ 
aircraft  artillery  (AAA)  site.  This  scenario  will  be  used  to  develop  the  ADS  test  environment  for 
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the  two  subsequent  test  phases  and  will  provide  the  baseline  data  for  comparison  with  the  ADS 
test  results.  Additionally,  the  performance  data  provided  a  baseline  for  attempting  correlation 
ability  across  all  three  phases  of  the  test.  The  Phase  1  test  scenario  was  very  structured  and 
constrained  to  provide  the  greatest  opportunity  for  repeatability  of  test  results  and  correlation 
across  the  test  phases.  For  instance,  the  flight  profile  consisted  of  a  single  aircraft  track  over  the 
ground  at  a  constant  altitude  and  ground  speed  through  the  limited  threat  environment.  The  rules 
of  engagement  (ROE)  for  the  threat  operators  were  somewhat  simple  and  limited  to  using  primary 
tracking  modes  without  electronic  counter-countermeasure  (ECCM)  options.  Also,  the  threats  of 
interest  were  commanded  on  and  off  at  specific  ranges  along  the  profile  so  the  actions  could  be 
repeated  as  consistently  as  possible  during  the  following  phases. 

Phase  2  will  be  a  test  using  a  real-time  digital  system  model  (DSM)  of  the  ALQ-131  receiver 
processor  linked  with  terminal  threats  at  the  AFEWES  facility  and  a  scripted  model  of  the 
terminal  threat  hand-off  portion  of  an  integrated  air  defense  system  (IADS).  The  threat  laydown 
from  the  OAR  baseline  will  be  replicated  in  the  synthetic  ADS  environment;  the  jammer  model 
will  be  flown,  via  the  scripted  flight  profiles  developed  from  the  actual  OAR  baseline  flights  and 
initial  HITL  test,  against  the  AFEWES  threats.  This  phase  will  ultimately  evaluate  the  ability  to 
apply  increased  fidelity  and  resources  through  ADS  early  in  the  development  cycle  of  a  jammer 
system  and  to  develop  and  refine  requirements  for  a  new  system  through  actual  effectiveness 
testing  of  a  digital  model  of  the  proposed  system. 

Phase  3  will  be  a  test  using  the  ALQ-131  jammer  installed  on  an  F-16  aircraft  located  in  an 
installed  system  test  facility  (ISTF)  at  the  Air  Combat  Environment  Test  and  Evaluation  Facility 
(ACETEF)  located  at  Patuxent  River  Naval  Air  Station,  Maryland.  This  facility  will  be  linked 
with  AFEWES  threats  using  the  same  threat  laydown  as  the  previous  tests  and  controlled  by  the 
same  scripted  flight  profile.  In  the  normal  EW  test  process,  ISTF  testing  is  used  late  in  the 
development  cycle  to  measure  the  effect  of  aircraft  systems  on  the  performance  of  the  jammer. 
This  type  of  testing  does  not  normally  provide  a  detailed  measure  of  the  effectiveness  of  the 
jammer  against  a  variety  of  threats.  This  test  will  not  only  evaluate  the  installed  system  in  the 
normal  mode  but  will  also  evaluate  the  ability  to  perform  closed-loop  effectiveness  testing  of  the 
jammer  installed  on  the  aircraft  prior  to  flight  test. 

1.3.2  EW  Test  Objectives 

It  is  difficult  to  assess  ADS  utility  in  the  real  world  of  EW  T&E.  Utility  is  more  easily  addressed 
if  you  can  quantify  cost  savings  or  if  you  can  show  a  capability  that  didn’t  exist  with  traditional 
test  methods.  Both  are  difficult  in  EW  testing.  There  are  significant  technical  challenges  in 
implementing  ADS  in  this  environment  as  well  as  programmatic  issues  such  as  cost  and  schedule 
impacts.  The  achievable  (not  just  theoretical)  performance  that  can  be  obtained  by  inserting  ADS 
into  the  established  EW  test  process  must  be  determined.  The  overall  objective  of  the  JADS  EW 
Test  is  to  address  these  questions  and  thus  assess  the  utility  of  ADS  to  EW  test  and  evaluation. 
Specific  test  objectives  are  listed  in  the  JADS  EW  APA  and  Program  Level  TAP/DMAP. 
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2.0  Phase  1  Overview 


2.1  Purpose 

2.1.1  Risk  Reduction  Test 

The  risk  reduction  effort  was  designed  to  collect  OAR  data  to  evaluate  procedures,  data  quality, 
and  analysis  techniques  in  preparation  for  the  OAR  baseline  test  that  collected  baseline  data  to 
answer  the  specific  ADS  and  ALQ-131  test  objectives.  Another  benefit  of  this  preliminary  flight 
test  phase  was  collection  of  data  that  aided  in  identification  of  test  range  and  aircraft 
instrumentation  limitations  as  well  as  data  reduction  system  software  anomalies.  This  risk 
reduction  allowed  the  test  team  to  restructure  some  test  events,  to  develop  and  optimize 
procedures  and  equipment  to  work  around  problems,  and  to  improve  the  data  reduction  software 
before  actual  data  collection  for  the  record.  Specific  performance  results,  configuration  changes, 
and  lessons  learned  from  the  six  missions  flown  over  the  WTR  are  delineated  in  Sections  3.1  and 
4.1. 

2.1.2  OAR  Baseline  Test 

The  purpose  of  the  open  air  test  was  to  establish  baseline  ALQ-131  performance  data  for 
subsequent  comparison  with  performance  data  collected  from  Phases  2  and  3  employing  ADS. 
Analysis  results  included  descriptive  statistics  on  jammer  measures  of  performance.  These  values 
and  those  from  the  HITL  testing  were  analyzed  to  support  a  decision  on  data  repeatability  to 
determine  whether  to  proceed  with  Phase  2  testing.  The  OAR  baseline  consisted  of  seven  flight 
missions  with  10.9  range  hours  flown  over  the  WTR  during  the  June  -  August  1998  timeframe. 
The  initial  mission  productivity  was  estimated  at  50%  and  actual  productivity  was  approximately 
78%. 

2.1.3  HITL  Test 

The  purpose  of  the  non-ADS  HITL  jammer  test  was  to  supplement  baseline  OAR  performance 
data  for  subsequent  comparison  with  performance  data  collected  from  tests  employing  ADS.  The 
HITL  test  was  not  part  of  the  Phase  1  test  baseline.  It  was  added  when  we  realized  we  would  be 
unable  to  collect  all  MOP  data  on  the  OAR.  The  integrated  product  team  (IPT)  chose  to  execute 
the  HITL  test  to  provide  the  missing  data  and  to  help  isolate  non-ADS  differences  in  the 
correlation  of  OAR  baseline  and  ADS-based  tests.  The  HITL  test  was  designed  and  executed  in 
about  half  the  time  normally  required.  Scheduling  was  forced  by  facility  workloads  and  JADS 
fixed  schedule.  This  test  provided  the  ability  to  collect  data  in  a  HITL  environment  for  ALQ-131 
measures  of  performance  that  were  not  able  to  be  collected  during  the  OAR  baseline  because  of 
instrumentation  limitations.  This  test  also  isolated  other  sources  of  variance  (such  as  radar  cross- 
section  [RCS]  and  threat  representation  such  as  missile  flyout)  between  the  AFEWES  facility  and 
the  WTR  and  characterized  such  sources  through  the  correlation  of  HITL  and  OAR  baseline 
results.  This  should  simplify  analysis  for  the  linked  test  phases  and  allow  better  definition  of  the 
impacts  of  ADS  on  the  test.  In  essence,  the  HITL  test  provided  a  bridge  between  the  OAR 
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baseline  results  and  the  Phase  2  and  3  results.  The  HITL  test  consisted  of  nine  days  of  testing  that 
produced  341  total  test  runs.  Usable  data  were  obtained  from  267  of  these  runs  resulting  in  an 
estimated  78%  productivity. 

2.1.4  SILTest 

The  purpose  of  the  non- ADS  SIL  jammer  test  was  to  collect  the  internal  timing  ALQ-131 
measures  of  performance  that  were  not  collected  on  the  OAR  baseline  or  the  HITL  tests.  The 
SIL  test  was  designed  and  executed  in  two  weeks  by  Georgia  Tech  Research  Institute  (GTRI). 
Scheduling  was  based  on  Air  Warfare  Center  (AWC)  making  the  Automatic  Multiple 
Environment  Simulator  (AMES)  facility  available  to  JADS  on  short  notice.  Existing  ALQ-131 
flight  qualified  test  instrumentation  prevented  these  measurements  from  being  made  during  the 
OAR  baseline  effort.  Time  synchronization  problems  prevented  the  measurements  from  being 
made  in  the  HITL  test.  The  SIL  test  was  our  last  attempt  at  measuring  this.  It  was  successful  and 
provided  us  with  the  response  data  we  needed  to  correctly  calibrate  the  digital  system  model  that 
will  be  used  in  the  Phase  2  test.  The  SIL  test  consisted  of  three  days  of  testing  that  produced  120 
total  test  runs.  All  of  these  data  were  usable. 

2.2  Organizational  Structure 

The  organizational  structure  for  coordinating  and  reporting  during  Phase  1  of  the  EW  Test: 
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2.2.1  Roles  and  Responsibilities 

2.2.1. 1  Deputy  Director,  Test  and  Evaluation  (DDT&E) 

•  Oversaw  the  JADS  JT&E 

•  Approved  JADS  financial  requirements 

•  Approved  the  program  test  plan  (PTP) 

•  Oversaw  the  analysis  and  reporting  of  test  results 

2.2.1. 2  JADS  JTF  and EW  Test  Team,  Albuquerque,  New  Mexico 

•  Conducted  overall  planning,  execution,  analysis,  and  reporting  of  the  test 

•  Managed  funding  to  accomplish  the  test 

•  Developed  and  evaluated  JADS  issues,  objectives,  measures,  and  related  data  elements 

•  Developed  and  integrated  the  components  of  the  EW  Test  environment 

•  Established  necessary  communication  links  with  test  participants 

•  Operated  the  Test  Control  and  Analysis  Center  (TCAC)  during  tests 

•  Reported  interim  and  final  results  to  OSD 

2.2.13  Western  Test  Range  (413th  Flight  Test  Squadron,  Edwards  AFB,  California) 

•  Provided  open  air  test  range  airspace  and  facilities 

•  Provided  test  mission  control  support 

•  Provided  ground-based  terminal  threat  radars  and  system  operators 

•  Provided  range  instrumentation  for  Phase  1  data  collection 

•  Provided  limited  reduced  threat  radar  system  data  for  JADS  EW  Test  analysis 

•  Published  a  post-test  mission  summary  report 

2.2. 1.4  Air  Force  Electronic  Warfare  Evaluation  Simulator  (AFEWES),  Fort  Worth,  Texas 

•  Provided  HITL  test  facilities  and  test  management  personnel 

•  Provided  overall  test  control  through  the  use  of  the  Tactical  Air  Mission  Simulator  (TAMS) 

•  Provided  simulated  terminal  threats  and  system  operators 

•  Provided  the  JammEr  Techniques  Simulator  (JETS)  for  background  signal  environment 

•  Provided  threat  test  management  centers  (TMC)  for  data  collection 

•  Provided  threat  instrumentation  and  electronic  and  hard  copy  data  products 

2.2.1. 5 Air  Warfare  Center  (AWC),  Eglin  AFB,  Florida 

•  Provided  system  integration  laboratory  (SIL)  facilities  to  collect  ALQ-131  response  time  data 

•  Provided  the  Automatic  Multiple  Environment  Simulator  (AMES)  and  operators 

•  Provided  a  synchronous  emitter  and  interfaced  it  with  the  AMES 

•  Provided  an  ALQ- 1 3 1  pod  (test  item),  pod  support,  and  pod  operation 
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•  Provided  the  ALQ-13 1  operational  flight  program  (OFP)  for  all  testing 

•  Executed  the  SIL  test  and  provided  JADS  with  resulting  data 

2.2.1.6  Air  National  Guard  Air  Force  Reserve  Test  Center  (AATC),  Tucson,  Arizona 

•  Provided  the  F-16  test  aircraft  (A/C)  and  ALQ-131  pods  for  OAR  and  HITL  testing 

•  Provided  the  Firefly  instrumentation  pods  for  OAR  testing 

•  Provided  test  aircraft,  aircrews  and  pod  maintenance  personnel  and  support 

2.3  Test  Approach 
2.3.1  Risk  Reduction 

The  EW  risk  reduction  test  was  initially  planned  employing  a  “racetrack”  flight  profile  through  the 
representative  threat  environment  with  a  trial  set  of  rules  of  engagement  and  test  conditions  to 
collect  EW  effectiveness  data.  As  risk  reduction  missions  were  flown,  refinements  were  made  to 
this  flight  profile  and  scenario,  and  a  final  set  of  conditions  was  chosen  to  employ  during  the 
remaining  OAR  baseline  test.  The  missions  were  flown  over  the  WTR  by  an  Air  National  Guard 
Air  Force  Reserve  Test  Center  (AATC)  F-16  test  aircraft  with  the  ALQ-131  jammer  and  airborne 
instrumentation. 

The  ALQ-131  was  employed  against  a  limited  threat  environment  consisting  of  threat  systems 
equivalent  to  the  Eglin  Test  Range  simulations  designated  as  the  Simulated  Air  Defense  System 
(SADS)  II,  SADS  III,  SADS  VI,  SADS  VIII,  and  Weapon  Evaluated  Simulated  Threat  (WEST) 
XI  (referred  to  as  WEST  X  during  the  test)  systems.  The  SADS  II  was  replaced  by  the  SADS  III 
after  the  first  mission  because  of  SADS  II  system  operating  problems  The  ALQ-131  was 
originally  programmed  with  modified  mission  software  tapes  provided  by  Warner  Robins  Air 
Logistics  Center  (WR-ALC),  Georgia,  to  receive  all  threats,  but  only  respond  to  our  threats.  This 
tape  set  failed  to  perform  correctly.  Our  second  attempt  was  a  full  combat  mission  software  tape 
to  receive,  process,  and  jam  an  extensive  threat  array  that  would  be  expected  in  an  actual  combat 
scenario.  This  tape  set  was  provided  by  AWC.  We  were  unable  to  use  this  tape  because  we  were 
unable  to  monitor  the  range  environment  sufficiently  to  reconstruct  correct  versus  incorrect  pod 
responses  during  analysis  and  to  allow  the  environment  to  be  reproduced  in  the  ADS-based  test 
phases.  Additionally,  Georgia  Tech  Research  Institute  suggested  a  different  electronic 
countermeasure  (ECM)  technique  that  would  allow  better  ADS  assessment  by  adding  more 
jammer/threat  interactions.  As  a  result,  JADS  authorized  GTRI  to  perform  limited  modeling  to 
determine  a  technique  that  would  be  appropriate.  AWC  supplied  a  custom  tape  set  to  meet  JADS 
requirements.  While  the  tape  set  was  constructed  of  off-the-shelf  techniques,  including  the 
recommended  WEST  X  technique,  JADS  did  not  select  the  most  effective  techniques  against 
each  threat.  The  tape  set  also  had  limited  receive  and  transmit  functions  tailored  to  the  specific 
four  threats  in  the  JADS  EW  Test  OAR  baseline  scenario.  This  simplification  limited  threat 
scenario  variability  and  maximized  repeatability,  aiding  correlation  assessment  during  later  test 
phases.  However,  this  simplification  affected  response  time  measures  of  performance,  making  any 
comparison  between  these  results  and  previous  results  from  other  tests  meaningless. 
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2.3.2  OAR  Baseline  Test 


The  OAR  baseline  test  approach  evolved  from  lessons  learned  during  risk  reduction.  The  final 
OAR  baseline  test  conditions  were  refined  during  the  earlier  risk  reduction  missions  and  consisted 
of  the  F-16  test  aircraft  flying  a  single  track,  north-south  flight  profile  through  the  threat 
environment  at  the  WTR.  The  stabilized  test  scenario  and  threat  operator  procedures  (ROE)  that 
had  been  developed  during  risk  reduction  were  employed  throughout  the  open  air  testing. 

2.3.3  HITL  Test 

The  HITL  test  was  generated  to  attempt  to  answer  two  MOPS  that  were  not  able  to  be  addressed 
during  OAR  baseline  testing  because  of  instrumentation  limitations  discussed  later.  These  MOPS 
were  1)  correct  threat  identification  response  time  and  2)  correct  ECM  technique  selection 
response  time.  The  HITL  test  used  one  of  the  same  two  ALQ-131  pods  flown  during  the  OAR 
baseline  testing  as  the  “hardware”  in  the  hardware- in-the  loop  during  the  test  at  AFEWES.  Sixty- 
two  OAR  baseline  mission  runs  were  scripted  and  executed  at  AFEWES,  replicating  the  OAR 
baseline  scenario  and  ROE  as  closely  as  possible.  The  simulated  threat  engagements  duplicated 
the  OAR  baseline  test  conditions  by  conducting  both  “dry”  (jammer  not  active)  and  “wet” 
(jammer  active)  runs.  Unlike  the  OAR  where  all  four  threats  were  engaged  at  the  same  time,  the 
AFEWES  threats  were  engaged  in  pairs  with  the  second  threat  pair  generated  using  the  JETS 
equipment  for  signal  density  only.  This  difference  should  not  have  a  measurable  effect  on  the 
measures  of  performance  (MOP)  results.  The  same  data  were  collected  as  those  collected  in  the 
OAR  baseline  to  answer  the  same  MOPs.  The  HITL  data  were  subsequently  compared  with 
OAR  baseline  data  for  correlation  purposes. 

2.3.4  SILTest 

The  SIL  test  was  created  and  executed  in  three  weeks  after  it  was  determined  that  the  HITL  test 
did  not  produce  adequate  data  to  answer  the  two  timing-related  MOPS  referenced  in  paragraph 
2.3.3.  JADS  was  able  to  take  advantage  of  an  offer  from  AWC  to  use  the  AMES  facility  at  no 
cost  to  JADS.  We  were  unable  to  transport  the  AATC-owned  pod  to  Eglin  AFB  in  time  for  the 
effort.  As  a  result,  JADS  used  a  pod  supplied  by  AWC.  The  pod  was  loaded  with  the  JADS 
tapes  for  the  test.  This  was  acceptable  to  JADS  as  the  pod-to-pod  variance  in  response  times  for 
directly  injected  signals  should  be  quite  small.  The  rules  of  engagement  and  the  geometry  of  the 
OAR  baseline  were  used  to  develop  a  set  of  computer  instructions  that  was  used  to  activate  the 
emitters  at  the  proper  time  in  the  flight  profile  and  to  condition  the  radio  frequency  (RF)  signals  to 
represent  the  correct  signal  strength  to  the  pod.  In  this  case,  we  were  only  interested  in  internal 
jammer  MOPs,  so  we  only  executed  wet  runs.  All  four  threats  were  active  and  controlled  by  the 
computer.  A  single  set  of  instructions  was  created  covering  both  the  north  and  south  runs.  In 
each  run,  the  aircraft  was  reversed  at  the  north  initial  point  (IP)  and  “flown”  back  to  the  south  IP . 
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2.4  Test  Objectives 

2.4.1  Risk  Reduction 

The  overall  objective  of  the  risk  reduction  test  phase  was  to  collect  ALQ-131  performance  data 
against  selected  ground  threats  to  evaluate  and  optimize  test  scenarios,  procedures,  data  quality, 
and  analysis  techniques  in  preparation  for  the  baseline  OAR  flight  test. 

2.4.1. 1  Specific  Mission  Objectives 

Table  2  summarizes  the  objectives  for  the  major  risk  reduction  mission  events. 

Table  2.  Risk  Reduction  Objectives 


Event 

Obi# 

Objective 

1.0  Refine  Phase  1 
execution  planning 

1.1 

Verify  OAR  test  support  concept  is  feasible 

1.2 

Implement/evaluate  the  rules  of  engagement 

1.3 

Verify  sufficient  information/experience  has  been  obtained  to 
prepare  Phase  1  documentation 

2.0  Gather  and 
evaluate  sample  data 

2.1 

Collect  and  evaluate  Firefly  data 

2.2 

Collect  and  evaluate  ALQ-131  pod  data 

2.3 

Verify  planned  sorties  will  produce  sufficient  sample  size 

2.4 

Demonstrate  ability  to  collect  repeatable  data  on  the  OAR 

2.5 

Verify  OAR  data  can  be  replicated  in  an  ADS  environment 

3.0  Validate  data 
processing  tools  and 
analysis  techniques 

Gather  engineering  data  to  verify  data  analysis  tools 

Exercise  data  analysis  path/process 

3.3 

Verify  analysis  techniques,  tools,  and  products 

4.0  Train  JADS 
personnel 

4.1 

Evaluate  observer  training  effectiveness 

4.2 

Evaluate  analyst  training  effectiveness 

5.1 

Complete  individual  mission  quick-look  reports  j 

Complete  final  risk  reduction  phase  report 

2.4. 1.2  Exit  Criteria 

The  risk  reduction  exit  criteria  are  directly  related  to  the  objectives  above.  Table  3  delineates  the 
exit  criteria  for  the  risk  reduction  phase. 
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Table  3.  Risk  Reduction  Exit  Criteria 


Objective 

Exit  Criteria 

1.1 

Sufficient  data  collected  to  confirm  that  OAR  test  support  concept  will  work 

1.2 

Rules  of  engagement  for  Phase  1  testing  are  stable  and  valid 

1.3 

Sufficient  information  obtained  to  prepare  quick-look  and  final  reports 

2.1 

Firefly  data  adequate  to  answer  appropriate  measures  of  performance 

2.2 

ALQ-1 3 1  data  adequate  to  answer  appropriate  measures  of  performance 

2.3 

Sufficient  samples  of  each  test  condition  to  credibly  answer  appropriate  MOPs 

2.4 

Data  are  consistent  and  repeatable  within  and  between  missions 

2.5 

OAR  data  collected  that  can  also  be  collected  at  AFEWES  during  Phase  2 

ES9 

Successful  data  analysis  using  ADRS  and  other  appropriate  tools 

Observers  able  to  understand,  collect,  and  document  all  required  OAR  data 

4.2 

Analysts  able  to  use  ADRS  and  other  tools  to  analyze  all  required  OAR  data 

5.1 

Quick-look  reports  completed  with  sufficient  detail  after  each  test  mission 

5.2 

Risk  reduction  final  report  completed 

2.4.2  OAR  Baseline 

The  overall  objective  of  the  OAR  baseline  testing  was  to  collect  baseline  ALQ-131  performance 
data  versus  the  threats  for  comparison  with  later  test  phases. 

2. 4. 2.1  Specific  Mission  Objectives 

Table  4  summarizes  the  specific  OAR  baseline  objectives. 

Table  4.  OAR  Baseline  Objectives 


Objective  # 

Objective 

1.1 

Establish  jammer  performance  in  an  OAR  environment 

1.2 

Establish  the  repeatability  of  OAR  baseline  test  results 

1.3 

Establish  ranges  of  OAR  baseline  statistics  for  event  data 

1.4 

Establish  range  of  correlation  coefficients  for  series  observables 

2. 4. 2. 2  Exit  Criteria 

The  OAR  baseline  exit  criteria  related  to  the  objectives  above  are  shown  in  Table  5. 


21 


Table  5.  OAR  Baseline  Exit  Criteria 


HS33S3301 

Exit  Criteria 

l.i 

Sufficient  OAR  baseline  data  collected  to  confirm  consistent  jammer  results 

1.2 

OAR  baseline  results  statistically  repeatable 

1.3 

Statistical  ranges  defined  for  all  event  data 

1.4 

Correlation  coefficients  defined  for  all  series  observables 

2.4.3  HITL 

2. 4. 3.1  Specific  Mission  Objectives 

Table  6  summarizes  the  specific  HITL  objectives. 

Table  6.  HITL  Objectives 


Objective 

Establish  jammer  performance  in  a  HITL  environment 

1.2 

Establish  the  repeatability  of  HITL  test  results 

1.3 

Establish  ranges  of  HITL  statistics  for  event  data 

1.4 

Establish  range  of  correlation  coefficients  for  series  observables 

2.43.2  Exit  Criteria 

The  HITL  exit  criteria  related  to  the  objectives  above  are  shown  in  Table  7. 

Table  7.  HITL  Exit  Criteria 


Objective  # 

Exit  Criteria 

1.1 

Collect  data  from  at  least  60  runs  executed  twice  (2  data  samples  per  run  = 

120  runs) 

1.2 

Use  the  high  level  architecture  (HLA)/runtime  infrastructure  (RTI)  interface 
logger  to  write  data  files  for  ADRS  processing 

1.3 

Isolate  ADS  impacts  from  other  factors  affecting  correlation 

1.4 

Provide  calibration  data  for  the  digital  system  model  (DSM)  for  Phase  2 

2.4.4  SIL 

The  only  objective  for  the  SIL  was  to  collect  60  samples  of  the  jammer  internal  timing  MOPs  for 
each  state  the  jammer  saw  in  a  perfect  engagement.  No  specific  entry  or  exit  criteria  were 
established  prior  to  beginning  the  effort.  GTRI  established  specific  technical  milestones  to  meet 
during  the  test  build-up.  These  were  related  to  specific  measurement  accuracy  and  integration 
requirements  that  had  to  be  satisfied  to  make  the  test  successful  for  JADS. 
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2.5  Methodology 


In  general,  the  test  method  employed  during  Phase  1  testing  was  structured  to  be  within  the 
constraints  imposed  to  provide  maximum  opportunity  for  repeatability  of  specific  events  and, 
therefore,  correlation  during  later  test  phases.  High  threat  density  and  multiple  reference  test 
conditions  that  would  be  typical  in  an  EW  operational  test  were  not  desirable  in  the  Phase  1  OAR 
baseline  and  HITL  testing.  A  single  reference  test  condition  against  a  limited  number  of  threats 
operated  within  strict  ROE  was  preferred.  After  several  risk  reduction  missions,  it  was  evident 
that  limited  relaxation  of  this  single  reference  test  condition  was  required.  Thus,  instead  of 
considering  all  northbound  and  all  southbound  runs  during  the  OAR  baseline  as  a  single  test 
condition,  runs  in  each  direction  were  separated  into  their  respective  data  sets.  The  specific  test 
conditions  and  configurations  that  were  accomplished  are  explained  in  the  following  paragraphs. 

2.5.1  System  Under  Test 

The  JADS  EW  Test  jammer  was  a  three-band  ALQ-131  Block  II  jammer  pod  modified  with 
instrumentation  interfaces  for  data  recording.  Two  jammer  pods  (serial  #0633  and  #0631)  were 
reserved  by  AATC  for  this  series  of  testing.  Both  pods  were  carried  on  the  two  test  aircraft 
during  the  first  two  risk  reduction  missions  and  only  one  pod  was  allowed  to  radiate  against  the 
threat  radars  at  any  one  time.  On  the  remaining  OAR  baseline  missions  only  one  aircraft  and  pod 
were  employed.  Pod  #0631  was  used  on  mission  #3.  Data  from  this  mission  indicated  problems 
with  the  pod  performance  and  subsequent  bench  tests  at  AATC  confirmed  that  the  high  band 
radiation  was  intermittent.  Thus,  pod  #0633  was  employed  on  missions  #4  through  #6  and 
remained  the  jammer  pod  for  all  of  the  following  OAR  baseline  tests.  Pod  #063 1  was  repaired 
and  used  for  the  HITL  testing  at  AFEWES. 

During  the  risk  reduction  and  OAR  baseline  testing,  ALQ-131  pod  data  were  obtained  from  two 
sources:  a  set  of  ALQ-131  recorder  cards  provided  by  the  Air  Warfare  Center,  Eglin  AFB, 
Florida,  and  a  military  standard  (MIL-STD)-1553  databus  interface  card  built  by  Southwest 
Research  Corporation  (SWR).  Both  sets  of  data  cards  were  carried  internal  to  the  pod.  The 
ALQ-131  recorder  cards  recorded  ALQ-13 1  receiver  processor  data  via  the  internal  digibus  in  the 
pod.  These  data  were  time-tagged  by  an  internal  clock  chip  and  saved  in  memory  on  the  cards  to 
be  retrieved  after  each  mission.  The  SWR  1553  databus  interface  card  collected  digibus  messages 
transmitted  by  the  jammer  pod  and  transmitted  the  data  to  the  Firefly  instrumentation  pod 
described  below.  For  the  HITL  testing,  the  ALQ-131  recorder  cards  were  replaced  by  a  digibus 
monitor  built  by  GTRI  to  collect  data.  The  Phase  3  test  will  also  use  the  digibus  monitor. 

The  ALQ-131  mission  and  preflight  software  programs  evolved  over  the  course  of  the  risk 
reduction  missions  and  two  integrated  product  team  (IPT)  meetings.  The  initial  IPT 
recommendation  was  to  use  a  limited  response  while  allowing  the  pod  to  try  and  identify  all 
signals.  This  WR-ALC  software  load  failed  to  perform  correctly  when  presented  with 
simultaneous  threats.  It  created  jamming  responses  that  were  not  supposed  to  be  part  of  the  tape 
load.  Our  second  approach  was  to  use  a  full  combat  mission  software  load  developed  by  AWC 
which  would  have  allowed  the  pod  to  receive,  process,  and  respond  to  all  valid  threat  signals  in 
the  environment.  This  program  was  designated  as  operational  flight  program  (OFP)  3130,  link 
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39,  preflight  message  (PT)/mission  tape  (MT)  3711.  Results  from  the  next  two  missions  indicated 
a  more  restrictive  software  load  was  required.  Limiting  the  pod’s  ability  to  receive  and  respond 
to  the  specific  threats  of  interest  to  JADS  would  provide  more  meaningful  results  and  better 
repeatability.  In  an  operationally  realistic  EW  effectiveness  evaluation,  it  would  be  desirable  for 
the  pod  to  process  the  entire  threat  environment;  however,  for  the  JADS  purpose  of  establishing  a 
baseline  test  condition,  it  was  more  useful  to  narrow  the  number  of  emitters  processed  to  a  few 
for  maximum  correlation  with  following  test  phase  results.  Also,  the  manually  tuned  Signal 
Analysis  System  (SASY)  RF  receivers  at  the  range  were  inadequate  to  capture  the  background 
environment  and  collect  samples  of  non-JADS  emitters  at  the  same  time.  AWC  strongly 
recommended  using  a  limited  reception/limited  response  tape  for  the  JADS  scenario  because  that 
matched  their  operational  practice  of  limiting  reception  and  response  to  the  expected  threat 
environment.  Several  technical  meetings  between  AWC,  GTRI,  and  JADS  resulted  in  a  final 
mission  tape  configuration  designated  PT-0700,  MT-0701.  This  tape  limited  the  pod  to  reception 
of  only  the  four  JADS  threats  of  interest  and  limited  jamming  responses  to  a  single  ECM 
technique  per  threat. 

2.5.2  Test  Scenario 

The  initial  risk  reduction  plan  called  for  six  F- 16  flight  test  missions  to  be  flown  at  the  WTR  with 
four  of  these  employing  the  ALQ-131  jammer  pod  and  two  missions  employing  the  ALQ-184 
jammer  pod.  The  latter  two  missions  were  planned  to  collect  data  for  AATC  evaluation  of  an 
upgrade  to  their  ALR-69  radar  warning  receiver  (RWR).  Because  of  late  delivery  of  the  software 
required  for  the  RWR  upgrade,  all  six  missions  to  collect  data  for  the  JADS  risk  reduction 
objectives  were  eventually  flown  with  the  ALQ-131. 

The  flight  profile  originally  planned  for  the  risk  reduction  and  later  OAR  baseline  missions  was  a 
50  nautical  mile  (NM)  “racetrack”  pattern  to  be  flown  between  14,000  to  20,000  feet  mean  sea 
level  (MSL)  at  450  knots  ground  speed.  This  scenario  was  modified  after  the  first  mission  to  a  22 
NM  “dog  bone”  profile  flown  at  9,000  feet  MSL  and  360  knots  to  allow  the  threats  better 
opportunity  for  full  acquisition  and  tracking  engagements.  The  planned  and  actual  mission 
profiles  are  illustrated  in  Figure  1.  The  racetrack  flight  profile  was  changed  to  a  “dogbone” 
profile  to  reduce  the  two-leg  flight  path  (two  conditions)  to  a  single  flight  path  through  the  threat 
environment  (one  condition).  It  was  also  shortened  to  22  NM  to  optimize  the  number  of  test  runs 
per  hour  while  still  allowing  the  four  threats  to  fully  engage  the  aircraft.  The  flight  altitude  was 
lowered  to  9,000  feet  MSL  to  bring  the  aircraft  within  firing  range  of  the  WEST  X  threat  system 
for  more  realistic  engagements.  Even  lower  altitudes  were  considered  but  rejected  to  avoid 
terrain  multipath  and  clutter  problems.  This  approach  was  taken  because  AFEWES  does  not  have 
a  terrain  or  ground  clutter  database.  Therefore,  correlation  of  results  between  test  facilities  is 
more  feasible  if  the  test  conditions  are  similar,  i.e.,  no  clutter.  All  OAR  baseline  missions  were 
flown  using  the  refined  profile;  the  HITL  and  SIL  tests  also  replicated  this  profile  for  consistency. 
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Initial  Profile  Final  Profile 


Figure  1.  Planned  and  Actual  Mission  Profiles 

The  final  threat  scenario  consisted  of  four  threat  systems  (two  command-guided  surface-to-air 
missile  sites,  one  semi-active  surface-to-air  missile  site,  and  one  anti-aircraft  artillery  site)  that 
used  the  Eglin  Test  Range  simulator  designations  SADS  III,  SADS  VI,  SADS  VIII,  and  WEST 
X.  The  threats  were  realistically  deployed  along  the  mission  profile  at  the  WTR  as  shown  in 
Figure  2. 


Figure  2.  JADS  EW  Test  Profde 
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2.5.3  Rules  of  Engagement 


To  ensure  consistent  threat  operating  procedures  and  the  maximum  opportunity  for  repeatability 
and  correlation  assessment  between  Phase  1  and  Phase  2  testing,  a  documented  list  of  ROE  was 
developed  prior  to  the  start  of  risk  reduction.  As  missions  were  flown  and  WTR  threat  operations 
became  better  understood,  modifications  were  made  to  the  ROE  for  each  threat.  The  ROE  were 
baselined  after  risk  reduction  mission  #3  and  the  same  guidelines  were  used  throughout  the 
remaining  missions.  These  same  rules,  with  slight  modifications,  were  also  used  at  AFEWES 
during  the  HITL  testing.  The  ROE  document  is  at  Appendix  B. 

2.5.3.1  OAR  Baseline/HITL/SIL  Differences 

The  IPT  identified  several  components  that  would  be  different  from  the  OAR  baseline  to  the 
ADS-based  test  phases.  These  differences  could  have  some  impact  on  the  EW  MOPs,  but  the 
impact  would  have  to  be  measured  or  estimated  in  order  to  isolate  them  from  the  impacts  of  ADS 
on  correlation.  The  HITL  test  was  added  to  help  identify  and  quantify  the  difference  between  the 
OAR  and  AFEWES  environments.  Some  of  the  expected  differences  associated  with  the  JADS 
EW  Test  and  the  facilities  involved  that  could  affect  correlation  are  shown  in  Table  8. 

Table  8.  Expected  Differences 


Test  Component 

Test  Location 

OAR  Baseline 
Versus  HITL 

HITL  Versus 
SIL 

OAR  Baseline 
Versus  ADS 

HITL  Versus 
ADS 

F-16  RCS 

X 

N/A 

X 

O 

ALQ-131  antenna  patterns 

X 

X 

X 

o 

Threat  representation 

X 

X 

X 

o 

ECM  technique 

X 

N/A 

X 

X 

Missile  flyout 

X 

N/A 

X 

o 

Time-space-position 

information 

(TSPI)/attitude 

X 

X 

X 

o 

RF  environment 

X 

X 

X 

o 

Legend:  X  =  Difference  Expected,  O  =  No  Difference 
Expected 


The  significant  differences  experienced  during  this  phase  of  testing  are  explained  in  the  following 
paragraphs. 

2.5.3. 1 . 1  Radar  Cross-Section  (RCS)  Differences 

The  actual  aircraft  RCS  on  the  range  was  different  from  the  RCS  database  at  AFEWES.  The 
AATC  aircraft  was  flying  a  nonsymmetrical  load  (one  Firefly  pod  on  station  7  only)  on  the  wings 
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during  the  OAR  baseline  test.  The  AFEWES  RCS  database  contains  data  that  support  an  aircraft 
equipped  with  a  symmetrical  load  (ordinance  on  both  stations  3  and  7).  RCS  was  not  used  in  the 
SIL. 

2.5.3. 1.2  Simulation  (Threat  Parameter  Baseline)  Differences 

Because  of  design  differences  between  the  AFEWES  and  the  OAR  simulators,  ROE  were 
developed  to  reduce  simulation  variability  between  facilities.  Some  of  the  differences  are  related 
to  functions  not  simulated  at  AFEWES  and  others  involve  radar  performance  tolerances.  For 
instance,  the  OAR  SADS  VI  includes  all  radar  functions  (target  acquisition,  tracking,  illumination, 
missile  seeker  etc.).  AFEWES  simulated  only  the  missile  seeker  portion  of  the  system.  AFEWES 
does  not  employ  any  optics,  television  or  identification  friend  or  foe  (IFF)  functions  to  aid  radar 
tracking,  whereas  the  OAR  systems  do  include  these  where  appropriate.  These  latter  features 
were  not  allowed  to  be  used  during  OAR  baseline  data  collection;  however,  optics  were  allowed 
during  initial  target  acquisition  to  ensure  the  target  was  in  radar  boresight  at  the  beginning  of  data 
runs.  This  capability  (to  find  the  target  unerringly)  was  simulated  at  AFEWES  by  starting  runs 
with  the  target  centered  (perfect  boresight)  in  the  radar  search  pattern.  There  were  other 
differences  between  the  OAR  and  AFEWES  systems  because  of  the  level  of  fidelity  in 
implementing  radar  capabilities.  Examples  include  differences  in  radar  beam  widths  (impacts 
clutter  elimination),  sidelobe  levels,  antenna  gain,  receiver  band  widths,  effective  radiated  power 
levels,  automatic  gain  control  levels,  and  implementation  of  features  such  as  moving  target 
indicator  and  home-on-jam.  The  specific  differences  in  these  areas  are  quantified  in  simulation 
validation  (SIMVAL)  reports  published  on  each  specific  threat  system  represented  at  both  the 
OAR  and  AFEWES.  The  threat  representations  at  the  SIL  were  simple  RF  generators 
programmed  to  create  the  threat  waveforms  at  the  proper  time  in  the  engagement.  The  emissions 
needed  to  be  accurate  enough  to  get  the  pod  to  respond.  No  other  verification  was  performed  on 
the  waveforms. 

2.5.3. 1.3  Test  Execution  Differences 

There  were  also  a  few  differences  in  the  test  execution  process  between  the  OAR  baseline  and 
HITL  tests.  The  major  difference  was  in  the  SADS  VI  operation  (mentioned  above),  where  the 
AFEWES  lab  only  simulated  the  missile  seeker  operation.  Therefore,  AFEWES  only  had  to 
provide  one  computer  engineer  to  control  the  signal  inputs  and  simulate  flyout  results  from  the 
seeker,  whereas  the  OAR  provided  a  full  operational  crew  to  control  all  of  the  radar  functions 
(target  acquisition,  tracking,  illumination,  missile  launch,  etc.).  Another  difference  was  how  the 
range  and  lab  provided  the  threat  radars  with  aim  point  information  to  the  aircraft.  The  range 
provided  a  slave  mode  for  the  radars  to  receive  pointing  angles  from  an  unjammed  range 
acquisition  radar,  while  the  lab  provided  this  function  by  giving  direct  target  location  to  the  threat 
simulator  (done  by  using  an  “initial  condition  (IC)  button”  or  a  “To  ECM”  switch).  The 
laboratory  provided  a  somewhat  clean  RF  environment  without  extraneous  signals,  so  the  lab  did 
not  require  a  SASY-type  facility  for  broadband  RF  monitoring.  For  execution,  the  lab  could 
dedicate  an  entire  day  for  one  project,  so  JADS  could  accomplish  60-80  runs  per  day  instead  of 
15-20  at  the  range.  The  lab  did  not  require  an  aircraft,  just  the  jamming  pod  in  an  RF  screened 
room.  The  pod  did  not  have  any  free  space  emanations,  but  instead  injected  the  attenuated  RF 
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signals  behind  the  antenna  feeds  in  the  pod.  Lastly,  the  lab  did  not  generate  new  aircraft  flight 
paths,  but  instead  used  the  same  time-space-position-information  (TSPI)  data  that  were  recorded 
at  the  range.  The  SIL  was  a  simpler  version  of  the  HITL  test.  No  operators  were  involved,  there 
was  no  issue  of  tracking  (all  threats  were  assumed  to  have  perfect  tracking)  and  all  signals  were 
injected  into  the  pod.  The  computer  controlled  the  activation  and  deactivation  of  the  RF  sources 
to  mimic  the  OAR  environment.  The  flight  path  was  a  computer-generated  straight  line  between 
the  two  IP  points  at  a  fixed  9000  feet  (ft)  MSL  altitude. 

2.5.4  Test  Configuration 

2.5.4.1  Risk  Reduction  and  OAR  Baseline 

2. 5.4. 1.1  Test  Range 

The  Western  Test  Range  provided  the  SAM  and  AAA  threat  environment,  threat  systems 
instrumentation,  RF  environment  monitoring,  TSPI,  and  data  collection  for  the  JADS  EW  Test 
risk  reduction  and  OAR  baseline  missions.  Specific  threats  and  their  parameters  are  listed  in  the 
range  Test  Support  Plan  dated  August  1997. 

2. 5.4. 1.2  Test  Aircraft 

Two  F-16C  Block  25  test  aircraft  were  initially  used  for  the  risk  reduction  missions.  Both  aircraft 
were  provided  and  flown  by  AATC.  Aircraft  tail  number  1180  was  a  two-seat  model  and  the 
second  aircraft  (tail  number  1262)  was  a  single-seat  model.  Both  aircraft  were  equipped  as  shown 
in  Figure  3  with  the  ALR-69  RWR  including  a  video  recorder,  Range  Applications  Joint  Program 
Office  (RAJPO)  pods  for  range  TSPI,  a  Firefly  instrumentation  pod,  two  fuel  tanks,  and  an  ALQ- 
1 3 1  Block  II  jammer  pod  as  the  system  under  test. 

Both  aircraft  were  flown  in  close  formation  on  the  first  two  risk  reduction  missions  primarily  to 
collect  data  for  AATC  equipment  upgrades  but  also  to  collect  the  JADS  EW  Test  data. 
Subsequently,  risk  reduction  missions  #3  through  #6  were  flown  with  a  single  aircraft.  The  desire 
was  to  fly  the  same  aircraft  on  all  risk  reduction  and  following  OAR  baseline  missions  to  maintain 
consistent  radar  cross-section  configuration;  however,  the  two-seat  aircraft  #1180  was  substituted 
for  one  mission  (mission  #4)  because  the  primary  test  aircraft  #1262  was  not  available.  Aircraft 
#1262  was  flown  for  all  remaining  OAR  baseline  sorties  during  Phase  1  testing. 

2 . 5 .4 . 1 . 3  Instrumentation 
2.5.4. 1.3.1  Test  Range 

Test  range  instrumentation  was  used  to  monitor  and  record  test  activity  and  ensure  data  were 
available  to  evaluate  the  JADS  EW  Test  measures  of  performance.  The  range  instrumentation 
consisted  of:  a  Signal  Analysis  System  and  associated  signal  vans;  spectrum,  pulse,  and  network 
analyzers;  individual  threat  site  instrumentation;  the  Radar  Detection  and  Performance  Analysis 
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System  (RDAPAS);  and  ground  tracking  reference  radar.  Detailed  descriptions  of  these  systems 
are  at  Appendix  E. 


2. 5.4. 1.3.2  Test  Aircraft 

Aircraft  data  were  collected  by  three  instrumentation  sources:  ALQ-131  recorder  cards  within 
the  pod,  the  Firefly  instrumentation  pod  carried  on  aircraft  station  #7,  and  video  recordings  of  the 
ALR-69  radar  warning  receiver  display.  The  ALQ-131  recorder  cards  recorded  ALQ-131 
receiver  processor  (R/P)  data  via  the  internal  digibus  in  the  pod.  These  data  were  time-tagged  by 
an  internal  clock  chip  and  saved  in  memory  on  the  cards  which  were  retrieved  after  each  mission. 
The  Firefly  pod  is  a  modified  general  instrumentation  pod  housing  the  GTRI-developed  Firefly 
recorder  and  a  global  positioning  system  (GPS)  receiver  and  antenna.  Data  collected  by  the 
Firefly  recorder  included  data  from  the  1553  interface  card  in  the  jammer  pod,  ALR-69  RWR 
threat  parameters,  and  inertial  navigation  system  (INS)  attitude  and  position  information.  The 
ALR-69  video  data  included  threat  situation  displays  showing  the  signal  environment  throughout 
each  mission.  These  systems  and  their  measurement  requirements  are  further  defined  below. 

2.5.4. 1.3.2. 1  ALQ-131  Recorder  Cards 

The  recorder  card  data  were  time-tagged  using  a  time-of-day  source  (synchronized  to  an  external 
source)  on  the  card  and  saved  in  memory  to  be  retrieved  after  the  missions.  The  recorder  card 
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was  required  to  capture  the  emitter  track  file  update  events  and  the  pod  responses  to  technique 
assignments.  These  data  were  used  to  determine  which  threats  had  been  detected  and  identified 
by  the  R/P  and  which  threats  had  been  assigned  a  jamming  technique.  The  time-tag  resolution 
and  accuracy  for  collected  events  was  required  to  be  less  than  10  milliseconds  (ms). 

2. 5.4. 1.3.2. 2  Firefly  Instrumentation  Pod 

The  Firefly  recorder  is  a  generic  MIL-STD-1553  bus  recorder  capable  of  monitoring  two  separate 
buses  simultaneously  in  its  standard  configuration.  The  recorder  contains  an  internal  GPS 
receiver  which  is  used  to  provide  position  data  as  well  as  a  universal  time  code  (UTC).  This  UTC 
is  used  to  time-tag  all  data  collected  by  the  Firefly  recorder.  For  the  JADS  test,  the  Firefly 
recorder  was  intended  to  collect  INS  data  from  the  avionics  bus  and  two  types  of  data  from  the 
EW  bus.  These  data  collected  from  the  EW  bus  included  data  from  the  ALQ-13 1  pod  transmitted 
by  the  1553  interface  card  and  from  the  ALR-69  RWR.  The  RWR  data  were  used  to  characterize 
the  RF  environment  as  seen  by  the  test  aircraft. 

The  measurement  requirements  for  the  Firefly  recorder  are  as  follows: 

•  Aircraft  INS  with  a  20  hertz  (Hz)  update  rate  and  a  latency  less  than  50  ms 

•  Record  all  EW  bus  traffic  with  minimal  errors  and  at  the  full  available  rate 

•  Time  tag  all  bus  data  with  a  UTC  time-tag  with  an  accuracy  and  resolution  less  than  10  ms 

2. 5.4. 1.3.2. 3  Southwest  Research  Corporation  1553  Interface  Card 

The  1553  interface  card  serves  as  an  interface  to  collect  digibus  messages  transmitted  by  the 
ALQ-13 1  pod  and  transmits  them  externally  over  a  MIL-STD-1553  serial  bus.  A  special  load  of 
the  operational  software  for  the  ALQ-13 1  jammer  pod  is  required  to  activate  this  instrumentation. 
Data  collected  over  this  interface  include  commanded  jammer  technique  assignments.  The  1553 
interface  cards  do  not  provide  a  time-tag  for  any  collected  data.  Therefore,  a  time-tag  must  be 
supplied  by  an  external  recorder  such  as  the  Firefly  recorder.  The  1553  interface  card  was 
required  to  collect  all  jammer  assignments  reported  by  the  ALQ-13 1  pod.  These  assignments 
included  a  list  of  technique  identifiers  assigned  by  the  pod.  Data  latencies  from  the  digibus  to  the 
external  MIL-STD-1553  bus  were  required  to  be  deterministic  or  exhibit  less  than  10  ms  variance 
to  be  acceptable  for  the  JADS  test  objectives. 

2.5.4. 1. 3.2.4  RAJPOPod 

The  RAJPO  is  a  wing-tip  mounted  pod  containing  an  internal  GPS  receiver,  an  INS,  and  a 
telemetry  transmitter.  This  instrumentation  used  in  conjunction  with  a  ground-based  GPS 
receiver  provides  an  accurate  position  update  for  the  aircraft  using  the  differential  GPS  approach. 
Position  and  attitude  data  from  the  pod  are  transmitted  to  a  telemetry  ground  station  and  merged 
with  a  ground-based  GPS  receiver  to  provide  an  estimate  of  GPS  position.  Position  data  are 
reported  in  latitude  and  longitude  coordinates  and  altitude.  A  position  update  is  required  once 
every  50  ms.  Aircraft  attitude  data  are  required  to  be  reported  at  the  same  update  rate  and  with  a 
precision  no  worse  than  the  test  aircraft’s  internal  INS. 
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2.5.4. 1. 3.2.5  ALR-69  Video  Recorder 


An  onboard  video  recorder  was  used  to  document  threat  indications  on  the  ALR-69  warning 
receiver  to  correlate  this  information  with  that  collected  at  the  ground  sites.  This  information 
included  threat  symbology,  approximate  azimuth  and  range  (based  on  power  level)  from  the 
aircraft,  threat  mode  changes,  and  jamming  indications. 

2. 5.4. 1.4  Go/No  Go  Criteria 

A  list  of  criteria  was  developed  after  risk  reduction  mission  #2  to  document  which  resources  were 
essential  to  conduct  the  risk  reduction  and  OAR  baseline  missions.  This  go/no  go  criteria 
addressed  resources  ranging  from  personnel  to  aircraft  and  range  systems  that  would  be  required 
to  conduct  or  continue  a  mission  if  certain  assets  failed  or  were  not  available.  The  list  of  go/no  go 
criteria  is  at  Appendix  C. 

2.S.4.2  HITL 

2. 5.4.2. 1  System  Under  Test  Configuration 

The  system  under  test  configuration  was  an  ALQ-131  Block  II  R/P  ECM  pod.  The  Digibus 
Traffic  Monitor  System  (DTMS)  replaced  the  ALQ-131  recorder  card  used  in  the  OAR  baseline 
phase.  (The  1553  interface  card  was  dropped  from  the  instrumentation  before  the  missions  for 
record  began.)  The  HITL  test  used  the  same  OFP,  the  same  limited  threat  detection  preflight 
message  tape  (PT),  and  the  same  limited  threat  response  message  tape  (MT)  as  used  in  the  OAR 
baseline  test. 

2.5.4.2.2  AFEWES  Simulator  Configuration 

Figure  4  illustrates  the  AFEWES  test  configuration  in  a  block  diagram  format. 
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Figure  4.  AFEWES  Test  Configuration 


2.5.4.2.3  Instrumentation 

2.5.4.2.3.1  Digibus  Traffic  Monitor  System 

The  ALQ-131  Digibus  Traffic  Monitor  System  (DTMS)  is  a  personal  computer  consisting  of  a 
Maxine  printed  wiring  assembly  (Maxine  card);  a  digibus  traffic  monitor  (DTM)  card  printed 
wiring  assembly;  an  enhanced  input/output  (I/O)  buffer  (EIOB)  module;  various  interconnecting 
cables  and  accessories;  and  specialized  software.  The  DTMS  was  used  as  a  digital  instrumentation 
system  for  the  jammer.  This  DTMS  was  integrated  into  the  HITL  test  configuration  for 
monitoring,  displaying,  and  recording  pod  digibus  traffic  in  real  time.  In  addition,  it  controlled 
and  nonintrusively  monitored,  displayed,  and  recorded  pod  operation  in  real  time. 

2.5.4.2.3.2  Aircraft 

The  aircraft  represented  in  the  HITL  was  a  composite  of  position,  attitude,  and  therefore,  RCS. 
Aircraft  position  and  attitude  were  recorded  from  the  OAR  TSPI  and  INS  data  and  played  back 
into  the  HITL  environment  in  the  form  of  a  script.  For  selected  threats,  the  script  played  back 
threat-centric  tracking  error  measured  in  the  OAR  baseline.  The  TSPI  position  and  attitude  as  a 
function  of  time  must  correspond  to  a  real  pass  made  in  the  OAR  baseline  to  have  maximum  value 
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for  correlation.  The  threat  simulators  responded  to  an  RF  signal  that  represented  a  theoretical 
reflection  of  the  aircraft.  The  signal  was  built  using  the  aircraft  position  and  attitude  in  relation  to 
the  threat  and  an  RCS  value  from  a  table.  The  signal  was  then  modified  to  account  for  its 
distance  and  motion  relative  to  the  threat  as  well  as  the  relationship  between  the  threat  antenna 
boresight  and  the  aircraft  position. 

2.5.4.2.3.3  Threat  Simulators 

The  AFEWES  closed-loop,  surface-to-air  weapon  system  simulation  for  the  HITL  test  consisted 
of  an  RF  head;  a  tracking  console;  a  software-programmable  antenna  pattern  generator  (SPAG) 
computer  with  customized  software  programs;  and  a  master  computer  which  provided  overall 
real-time  control  of  the  target  tracking  simulation.  The  RF  head  simulates  real-time  echo  signals. 
It  also  modulates  and  scales  RF  signals  to  simulate  the  effects  of  range,  antenna  gain  patterns,  and 
other  factors  in  the  radar  range  equation.  The  tracking  console  contains  receivers,  target  tracking 
servocontrol  systems,  a  system  synchronizer,  simulation  displays,  and  man-in-the-loop  radar 
operator  controls.  Antenna  patterns  were  simulated  on  the  SPAG  computer  through  a  table  look¬ 
up  process.  The  simulators  used  during  this  test  were  the  SADS  III,  SADS  VI,  SADS  VIII,  and 
WEST  X. 

2.5.4.2.3.4.  Tactical  Air  Mission  Simulator  (TAMS) 

The  TAMS  data  represented  the  aircraft  platform  consisting  of  TSPI,  attitude,  RCS,  and  the  pod 
antenna  patterns  produced  by  the  target  generation  software  designated  as  the  “platform  federate” 
in  the  ADS  phases  of  the  EW  Test.  Additionally,  there  was  graphic  user  interface  (GUI)  software 
that  depicted  the  real-time  aircraft  flight  profile  and  distances  from  the  respective  IP — south  for 
northbound  flights  and  north  for  southbound  flights.  The  AFEWES  software  (threat  simulations), 
developed  to  Phase  2  specifications,  collected  raw  HITL  data  and  formatted  the  data  according  to 
the  JADS  Phase  2  Interface  Control  Document  (ICD).  These  data  were  collected  using  a  JADS 
runtime  infrastructure  (RTI)  interface  logger.  RTI  logger  files  were  processed  into  a  comma- 
delimited  format  to  provide  input  files  for  post-test  analysis  using  the  Automated  Data  Reduction 
Software  (ADRS)  tool. 

2.5.4.2.3.5  Test  Management  Centers 

The  test  management  centers  were  used  to  collect  data  from  the  closed-loop  simulators — the 
jamming-to-signal  ratio  (J/S),  radar  tracking  error,  and  missile/projectile  miss  distance.  Raw  data 
collected  during  the  HITL  test  were  available  for  evaluation  via  digital  strip  chart  printouts, 
graphics,  miss  distance  calculations,  and  radar  scope  video  tapes. 

2.5.4.2.3.6  JammEr  Techniques  Simulator 

The  AFEWES  JammEr  Techniques  Simulator  (JETS)  was  used  to  provide  background  signals  for 
the  HITL  test.  Specifically,  during  the  generation  of  closed-loop  simulations  of  the  SADS  VIII 
and  the  WEST  X,  JETS  replicated  the  RF  signals  of  the  SADS  III  and  SADS  VI  emissions. 
During  the  generation  of  closed-loop  simulations  of  the  SADS  III  and  SADS  VI,  JETS  replicated 
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the  RF  signals  of  the  SADS  VIII  and  WEST  X  emissions.  For  the  HITL  test  JETS  did  not 
generate  ECM  techniques  (in  accordance  with  the  ROE  for  the  Phase  1  baseline  test). 

2.5.4.2.3.7  Spectrum  Analyzers 

Spectrum  analyzers  provided  a  real-time  qualitative  assessment  of  jammer  responses  and  threat 
emissions.  Their  primary  function  was  to  assess  the  threat  simulators  health  during  the  runs  to 
identify  major  problems  that  may  be  corrected  quickly.  The  measurement  requirement  for  the 
spectrum  analyzers  was  to  provide  visual  displays  of  emitter  and  ECM  waveforms  for  real-time 
verification.  Selected  spectrum  analyzer  displays  were  recorded  in  the  TMC  to  assist  in  post-test 
analysis. 

2. 5.4. 2.3. 8  AFEWES  Facility  Gateway  Software  (HLA/RTI  logger) 

The  AFEWES  gateway  was  composed  of  a  high  level  architecture  (HLA)  interface  and  interfaced 
to  the  models  and  simulations  within  the  AFEWES  facility.  This  is  not  part  of  a  standard  HITL 
test.  JADS  selected  the  logger  as  the  primary  data  collection  point  to  avoid  changing  data 
analysis  software  to  read  standard  HITL  data  formats.  It  was  also  implemented  to  reduce  risk  in 
Phase  2  execution.  The  logger  resided  in  the  interface  between  the  AFEWES  gateway  and  the 
RTI.  The  logger  software  recorded  all  function  calls  to/ffom  the  RTI  along  with  all  of  the 
functional  data  parameters.  The  AFEWES  gateway  was  set  up  to  publish  aircraft  platform  data 
consisting  of  altitude,  attitude,  roll,  pitch  and  heading,  velocity,  location,  and  orientation  even 
though  there  were  no  other  federates  subscribing  to  the  data.  This  did  not  affect  the  logger,  since 
the  data  were  passed  to  the  logger  before  they  were  passed  to  the  RTI.  When  the  federate 
published  data,  it  called  the  RTI  “updateAttributeValues”  function.  Another  communication  step 
was  required  with  the  logger-in- the-loop.  When  the  logger  was  linked  with  the  federate,  the 
federate  called  the  logger  “updateAttributeValues”  function.  The  logger  stored  the  function 
identification  and  data  in  the  log  file  buffer  and  then  called  the  RTI  “updateAttributeValues” 
function.  When  a  log  file  buffer  became  full,  it  was  written  asynchronously  to  the  log  file  and  a 
new  buffer  was  created.  Upon  completion  of  the  HITL  test,  the  log  files  were  read  and 
formatted  with  various  custom  programs  to  prepare  the  files  for  playback  in  ADRS.  The 
create_adrs_file  program  read  the  log  file  data  and  created  a  comma-delimited  text  file  that  can  be 
read  by  ADRS. 

2.5.5  Constraints  and  Limitations 

During  the  Phase  1  test  design,  JADS  developed  the  EW  Test  by  applying  a  set  of  goals  and 
constraints  relative  to  test  content,  cost,  schedule,  and  personnel  described  in  the  EW  APA.  The 
net  result  of  all  the  constraints  is  that  the  JADS  EW  Test  represents  a  developmental  test. 
However,  none  of  the  constraints  in  this  phase  prevent  more  dynamic  tests  from  being  replicated 
in  an  ADS  environment.  The  major  Phase  1  procedural  constraints  are  listed  in  the  bulleted 
paragraphs  below.  Other  limitations  related  to  program  cost,  schedule,  and  personnel  follow. 

•  Phase  1  testing  was  restricted  to  a  single  test  condition  that  could  be  replicated  by  equipment 
and  facilities  used  in  the  later  test  phases.  This  was  done  to  reduce  variability  of  test 
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conditions  to  aid  statistical  correlation  of  results.  Such  a  restriction  would  not  be  acceptable 
for  an  operational  test  designed  to  measure  EW  effectiveness,  but  aided  in  the  assessment  of 
ADS  impacts. 

•  The  OAR  threat  simulators  were  limited  to  four  that  AFEWES  could  replicate  for  Phase  2  and 
3  comparisons. 

•  JADS  restricted  both  the  detection  and  response  capabilities  of  the  jammer.  This  reduced  the 
variability  in  the  response  time  of  the  ALQ-131  and  the  potential  for  erroneous  threat 
identification  or  ECM  technique  selections. 

•  To  avoid  multipath  transmissions  and  ground  clutter,  the  test  aircraft  minimum  altitude  was 
restricted  to  9,000  feet  MSL  for  all  data  runs. 

•  ROE  restrictions  were  imposed  on  the  OAR  threat  operators  to  standardize  system  operation 
as  much  as  possible  for  replication  at  AFEWES. 

•  To  repeat  the  engagements  from  Phase  1  as  closely  as  possible  in  later  phases,  each  site 
operator’s  engagement  windows  were  scripted. 

•  The  same  test  aircraft  were  used  on  all  OAR  baseline  missions  to  limit  likely  variations  in 
target  RCS.  However,  during  two  OAR  baseline  missions  we  were  unable  to  secure  a  RAJPO 
pod  for  each  wing  tip. 

•  Jamming-to-signal  ration  (J/S)  data  were  no  able  to  be  collected  on  the  OAR.  No  adverse 
impact  was  evident  since  J/S  information  was  subsequently  calculated  using  HITL  test  data. 

2. 5. 5.1  Cost 

JADS  had  an  established  test  budget  and  the  Phase  1  test  was  developed  within  the  budget.  The 

resulting  design  represented  the  minimum  jammer  test  required  to  evaluate  the  utility  of  applying 

ADS  to  EW  T&E.  The  costs  shown  in  Table  9  were  incurred  during  the  execution  of  this  test 

phase.  These  costs  include  the  unique  instrumentation  required  to  execute  the  test. 
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Table  9.  Phase  1  Cost  Summary 


Cost  Item 

Amount 

AATC  (contractor  salaries  and  travel  costs) 

$60,000 

Note:  $4  thousand  (K)  per  flying  hour  was  saved  by  using  AATC  resources 

Western  Test  Range 

$351,500 

Total 

•  Risk  Reduction  Phase 

$120,300 

•  OAR  Baseline  Phase 

$231,200 

AFEWES 

$636,946 

Total 

•  HITL  Planning  and  Execution 

$515,593 

•  SADS  VI  Missile  Flyout  Planning  and  Execution 

$121,353 

Georgia  Tech  Research  Institute 

$924,797 

Total 

•  Test  Support 

$202,000 

•  RDAPAS  and  SASY  Analysis  Software 

$24,238 

•  Automatic  Data  Reduction  System  Modification 

$314,043 

•  OAR  Analysis  Software 

HEHHI 

•  ECM  Technique  Analysis 

•  Pretest  Analysis 

HE2E3 

•  ALQ-131  Instrumentation 

$161,000 

•  1553  Interface  Cards  and  Firefly  Hardware 

$65,000 

JADS  Travel  Note:  JADS  EW  Test  team  travel  costs  were  not  tracked 

* 

separately  within  the  overall  JADS  budget. 

2. 5. 5.2  Schedule 

The  primary  schedule  constraint  associated  with  the  Phase  1  effort  was  related  to  the  schedule  of 
the  overall  JADS  EW  Test  program.  The  program  has  a  finite  end  date  and  the  risk  reduction  and 
OAR  baseline  missions  were  the  basis  for  all  subsequent  testing.  The  flight  testing  and  two-week 
AFEWES  effort  had  to  be  completed  by  30  September  1998  so  as  not  to  impact  the  start  of  Phase 
2  testing.  The  original  schedule  optimistically  called  for  risk  reduction  to  be  complete  by 
November  1997.  The  problems  typically  associated  with  EW  flight  testing  precluded  completion 
until  May  1998;  some  adjustments  to  the  remaining  flight  test  and  HITL  test  schedules  were 
required  during  Phase  1.  Risk  reduction  missions  could  not  be  flown  back-to-back  because  it  was 
necessary  to  retrieve,  reduce,  analyze,  and  understand  data  from  each  sortie  to  make  changes 
prior  to  the  next  sortie — the  primary  function  of  risk  reduction.  In  some  cases,  new  software 
programs  had  to  be  loaded  to  optimize  ECM  techniques,  and  mission  results  sometimes  required 
other  time-consuming  adjustments  prior  to  the  next  mission.  OAR  baseline  flights  were 
temporarily  suspended  in  late  July  1998  for  the  two-week  HITL  effort  because  the  same  ALQ- 
131  pods  were  required  for  both  tests.  OAR  baseline  missions  resumed  in  August  1998  and  were 
completed  on  20  August  1998. 
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2. 5. 5.3  Personnel 


Another  constraint  imposed  on  the  Phase  1  test  was  to  conduct  the  test  program  using  current 
JTF  assigned  personnel  augmented  by  experienced  contractor  and  test  facility  personnel.  Because 
of  the  relatively  short,  fixed  period  of  the  JTF,  the  initial  cadre  of  military  personnel,  in  many 
cases,  was  not  replaced  as  some  departed  for  other  assignments.  Additional  contractor  personnel 
were  used  to  supplement  some  of  these  losses,  but  overall  JADS  test  team  manning  to  accomplish 
the  necessary  tasks  was  limited  throughout  the  test  and  reporting  period. 

Personnel  considerations  at  the  test  sites  also  created  inherent  limitations.  The  WTR  threat 
operators  supported  other  tests  during  the  period  of  the  JADS  flight  test  and  therefore  worked 
shifts.  As  a  result,  the  same  operators  were  not  always  available  to  operate  the  same  threat 
systems  for  each  JADS  mission.  This  changeover  required  JADS  observers  to  rebrief  the  threat 
operators  prior  to  each  mission  on  JADS-unique  rules  of  engagement,  test  objectives,  and  other 
JADS-unique  aspects  in  order  to  maintain  operating  consistency  on  all  missions.  A  limited 
number  of  personnel  (threat  operators)  at  AFEWES  during  the  HITL  effort  was  a  contributing 
factor  in  determining  the  total  number  of  threats  that  could  be  operated  simultaneously.  The 
desire  was  to  have  all  four  threat  systems  up  at  the  same  time  to  be  consistent  with  the  WTR 
operation.  However,  the  systems  were  operated  in  pairs  by  the  AFEWES  operators  while  low 
fidelity  simulation  of  the  remaining  two  systems  emissions  were  injected  as  background 
environment. 

2.5.S.4  Threat  Environment 

2. 5. 5.4.1  Risk  Reduction/OAR  Baseline 

The  range  threat  environment  for  both  risk  reduction  and  OAR  baseline  flight  testing  were  the 
same  and  represented  very  realistic  systems  with  some  limitations.  An  IADS-type  environment 
could  have  been  used  to  add  realism  to  the  environment  but  was  dropped  from  the  JADS  test 
design  for  both  cost  and  test  simplicity  considerations.  The  same  was  true  for  threat  density. 
More  terminal  threats  were  available  but  not  employed  in  the  interest  of  keeping  the  AFEWES 
duplication  of  the  range  environment  manageable.  Of  the  four  terminal  threats  employed,  the 
SADS  VI  had  the  most  significant  test  limitations  on  the  OAR.  This  semi-active  missile  system 
was  employed  with  very  realistic  acquisition,  target  track,  and  illuminator  radar  implementation; 
however,  the  AFEWES  missile  simulation  was  added  to  provide  higher  fidelity  miss  distance  data. 
This  combination  provided  JADS  with  the  highest  fidelity  simulation  for  that  threat  system.  This 
caused  the  engagements  to  be  run  twice,  once  on  the  OAR  and  once  at  AFEWES,  to  get 
complete  engagements  and  produce  miss  distance.  Cost  limitations  caused  us  to  drop  the  actual 
shots  taken  on  the  OAR  and  use  scripted  shots  at  AFEWES.  The  script  was  based  on  the  average 
firing  range  for  valid  shots  taken  at  the  OAR.  The  other  three  threats  had  no  operational 
limitations  significant  to  the  JADS  test. 
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2. 5. 5.4.2  HITL 


The  HITL  threat  environment  was  representative  of  the  conditions  set  forth  during  the  live  flight 
portion  of  the  OAR  baseline  testing.  Although  on  the  surface  the  circumstances  at  AFEWES 
were  different,  the  ALQ-131  stimulation  was  comparable  to  all  four  simulators  being  active 
throughout  the  engagement  run.  The  runs  were  set  up  using  only  two  fully  active  threats  with 
operators  because  of  personnel  limitations.  The  other  two  threats  had  their  respective  signals 
replicated  by  the  JETS.  Three  of  the  four  threats  were  folly  manned  for  at  least  a  week.  The 
SADS  VI  only  simulated  a  missile;  there  were  no  operators  involved.  Additionally,  realistic 
flyout  models  were  used  for  the  missile  engagements. 

25.5.5  Instrumentation 

2. 5. 5. 5. 1  Risk  Reduction/OAR  Baseline 

The  various  range  and  aircraft  instrumentation  systems  generally  provided  adequate  data  to  assess 
the  JADS  MOPs  with  some  limitations.  The  major  aircraft  limitations  are  noted  in  the  following 
paragraphs.  Detailed  range  instrumentation  limitations  are  listed  in  Appendix  E.  Once  the 
limitations  were  known,  JADS  determined  that  all  of  the  known  and  expected  variance  sources 
between  the  OAR  baseline  and  the  ADS-based  tests  could  not  be  resolved.  Therefore,  JADS 
designed  and  executed  a  HITL  test  to  isolate  the  ADS-induced  variance  from  the  facility 
differences. 

2. 5. 5. 5. 1.1  ALQ-131  Recorder  Cards 

The  recorder  cards  provided  the  correct  data  types  and  the  emitter  track  file  updates,  and 
jamming  assignments  were  collected  successfully.  The  time  resolution  was  adequate;  however, 
the  accuracy  was  unacceptable  during  all  test  missions.  JADS  was  unable  to  accurately 
characterize  the  time  drift  over  a  single  mission  of  the  recorder  card  time  source.  It  varied  by 
several  seconds  from  one  mission  to  the  next.  The  initial  time  injection  was  unreliable  as  well. 
Initially,  the  method  used  to  perform  this  injection  was  to  enter  a  time  on  a  personal  computer 
(PC)  keyboard  based  on  a  time  display  seen  on  a  hand-held  GPS  receiver.  This  method 
introduced  errors  on  the  order  of  one  second.  A  direct  injection  method  was  attempted  using 
equipment  provided  by  AWC.  Unfortunately,  however,  it  was  determined  that  this  equipment 
was  faulty  and  could  not  be  used.  Because  of  these  problems,  timing  response  data  were  not 
collected  on  the  range.  However,  the  timing  MOPs  were  so  important  that  this  lack  of  data  was 
the  genesis  of  the  HITL  test.  The  timing  MOPs  were  subsequently  evaluated  at  AFEWES  during 
the  HITL  test  and  finally  during  the  SIL  test  at  AWC. 

2.5. 5. 5. 1.2  1553  Interface  Cards 

Significant  limitations  with  these  interface  cards  made  them  unusable  for  JADS  purposes.  The 
1553  interface  provided  inaccurate  data  with  intolerable  latencies.  The  software  required  to 
report  jammer  activity  to  the  1553  interface  was  executed  at  the  lowest  priority  in  the  pod 
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software.  This  contributed  to  data  collection  latency  that  varied  from  3  to  12  seconds.  In  some 
instances  when  the  number  of  active  threats  exceeded  one,  data  were  lost  entirely  over  this 
interface.  In  addition,  the  1553  recording  process  was  intrusive  to  normal  pod  operations  because 
of  the  way  the  jammer  operational  software  reported  jammer  technique  data.  Prior  to  learning 
this,  JADS  contracted  with  GTRI  to  build  a  set  of  these  cards.  After  getting  the  drawing  package 
from  WR-ALC,  GTRI  fabricated  the  cards,  but  they  did  not  pass  functional  checks  performed  by 
GTRI. 

2.5. 5. 5.2  HITL 

2. 5. 5. 5.2.1  Digibus  Traffic  Monitor  System 

The  ALQ-131  Digibus  Traffic  Monitor  System  displayed  ALQ-131  R/P  responses  to  the  threat 
simulators  in  real  time  -  both  threat  detection  and  mode  change  activity.  The  DTMS  was  synched 
to  an  Inter-Range  Instrumentation  Group  (IRIG)-B  time  source,  and  it  provided  accurate  jammer 
activity  files  to  within  one  ms.  A  jammer  activity  file  recorded  the  jammer’s  threat  activation, 
jammer  threat  identification  and  jammer  response  times.  These  files  were  one  of  two  input  files 
required  to  generate  the  HITL  data.  The  other  source  was  the  RTI  interface  logger  files  collected 
within  the  AFEWES  gateway. 

2. 5. 5. 5.2.2.  Tactical  Mission  Simulator 

The  TAMS  graphically  displayed  the  simulated  aircraft  throughout  each  engagement  run  while  the 
recorded  aircraft  composite  information  was  played  back  at  20  Hz.  Occasionally,  the  aircraft 
stopped  moving  along  the  intended  flight  path,  but  normally,  the  aircraft  script  (flight  profile) 
loaded  and  the  aircraft  flew  as  scripted.  After  aborted  runs  for  any  type  of  failures,  subsequent 
reruns  of  the  script  were  successful.  TAMS  also  functioned  as  the  test  control  station  during  each 
run.  The  AFEWES  test  controller  used  the  displayed  range  from  IP  to  identify  state  changes  of 
the  simulators — the  “on”  and  “off’  commands.  This  procedure  worked  well  as  long  as  the  test 
controller  was  attentive  in  following  the  site  controller’s  matrix  and  observing  the  range  changes 
on  the  computer  display. 

2. 5. 5. 5.2.3.  Test  Management  Centers 

The  TMC  threat  simulator  parameter  strip  charts  were  an  invaluable  real-time  analysis  tool.  The 
capability  to  view  printouts  of  tracking  error  proved  beneficial  in  troubleshooting  algorithms  used 
to  calculate  aircraft  position  at  the  time  of  engagement — the  simulator’s  view  of  the  engagement. 
Additionally,  the  strip  charts  provided  direct  feedback  for  a  “valid”  engagement.  The  TMC  also 
depicted  a  two-dimensional  view  of  the  engagement  and  recorded  the  simulator’s  threat  scopes. 
However,  there  was  a  miscommunication  or  misinterpretation  of  the  “common  time”  required  to 
facilitate  post-test  analysis.  JADS  required  that  all  data  sources,  including  video,  have  a  single 
reliable  time  reference.  Given  our  architecture  within  the  AFEWES  facility,  they  were  not 
adequately  set  up  to  fulfill  our  request.  Although  AFEWES  had  the  capability,  the  TMCs  were 
not  wired  to  handle  a  time  source.  However,  by  the  second  day  of  testing,  GPS  time  was 
transmitted  to  each  recorded  display.  AFEWES  placed  a  camera  in  front  of  the  GPS  display  and 
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sent  the  video  (of  the  time)  to  the  TMCs.  They  superimposed  the  time  video  onto  the  recorded 
displays.  This  method  proved  to  be  a  workable  alternative. 

2. 5. 5. 5.2.4.  JammEr  Technique  Simulator(JETS) 

JETS  simulated  the  RE  signals  of  the  nonparticipating  threats  for  the  two  weeks  of  testing.  A 
spectrum  analyzer  monitored  the  waveforms  for  proper  tum-on  and  turn-off  times  and  for  the 
appropriate  RF  signal  during  each  run. 

2. 5. 5. 5.2.5.  AFEWES  Software 

Upon  post-test  review  of  the  HITL  data,  there  were  numerous  data  elements  consistently  missing 
from  the  runs  and  a  problem  with  the  relative  location  of  the  aircraft  position.  The  AFEWES 
software  did  not  publish  the  prescribed  modes  and  codes  detailed  in  the  ICD.  The  modes  and 
codes  identified  the  threat  systems  and  the  current  state  of  each.  These  data  were  transmitted 
from  the  threats  correctly,  but  the  software  failed  to  accept  the  threat  states  and  publish  the 
corresponding  modes  and  codes.  These  shortfalls  were  corrected,  and  the  recorded  HITL  data 
from  the  AFEWES  optical  disk  format  were  reprocessed  through  the  amended  software  and 
recorded  by  the  RTI  interface  logger. 

2.6  Scheduling 

2.6.1  Risk  Reduction 

The  original  risk  reduction  schedule  called  for  six  missions  to  be  flown  between  August  and 
November  1997.  Flight  testing  did  begin  on  27  August  1997  with  the  first  mission;  however, 
technical  problems  with  the  ALQ-131  OFP,  instrumentation,  and  ECM  technique  response 
software  programs  delayed  the  second  mission  until  18  November  1997.  Similar  problems 
combined  with  test  range  and  AATC  scheduling  conflicts,  asset  nonavailability,  and  data  delivery 
problems  continued  to  cause  delays  in  mission  scheduling  throughout  the  risk  reduction  phase. 
The  sixth  and  final  risk  reduction  mission  was  flown  on  20  May  1998. 

Table  10  summarizes  the  chronology  of  the  major  scheduled  events  during  this  portion  of  Phase  1 
testing. 
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Table  10.  Risk  Reduction  Schedule  History 


Mission 

Date 

Flown 

Canceled 

Comments 

28  Aug  97 

1  Hr. 

1st  risk  reduction  sortie.  2  F-16s  versus  SADS  II,  III,  VI,  VIII  and  WEST  X. 

1  wet  and  2  dry  runs.  Only  mission  against  SADS  II.  AATC  profile  also 
flown. 

15  Sep  97 

X 

Late  data  receipt  (8  Oct)  from  the  WTR  for  28  Aug  mission  caused  JADS 
cancellation. 

7  Oct  97 

X 

WTR  had  range  time  available,  but  test  aircraft  was  unavailable. 

18  Nov  97 

2  Hr. 

2nd  risk  reduction  sortie.  2  F-16s  versus  SADS  III,  VI,  VIII  and  WEST  X. 

14  wet  and  4  dry  runs. 

3  Dec  97 

X 

Late  data  receipt  (15  Dec)  from  the  WTR  for  18  Nov  mission  caused  JADS 
cancellation. 

28  Jan  98 

1  Hr. 

3rd  risk  reduction  sortie.  1  F-16  (A/C  #1262)  versus  4  threats. 

8  wet  and  4  dry  runs. 

17  Feb  98 

X 

Originally  scheduled  by  the  range  but  canceled  due  to  a  higher  priority 
mission 

25  Feb  98 

1  Hr. 

4th  risk  reduction  sortie.  1  F-16  (A/C  #1180)  versus  4  threats. 

7  wet  and  4  dry  runs. 

17  Mar  98 

X 

Scheduled  range  time  canceled;  new  ALQ-131  mission  tape  not  ready.  Also 
A/C  #1180  had  delaminated  canopy  and  A/C  #1262  was  in  phase  inspection. 

19  Mar  98 

20  Mar  98 

2  Apr  98 

7  Apr  98 

X 

These  dates  had  available  range  times  scheduled  at  the  WTR  but  were  never 
placed  on  the  AATC  mission  schedule.  An  Air  Combat  Command  project 
order  required  for  AATC  to  fly  JADS  missions  was  in  coordination  from 

1 1  Mar-8  Apr  98. 

21  Apr  98 

X 

Scheduled  by  range;  not  placed  on  AATC  schedule  -  range  time  was  too 
early  (0700-0900)  for  AATC  to  support. 

28  Apr  98 

1.5  Hr. 

5th  risk  reduction  sortie.  1  F-16  (A/C  #1262)  versus  4  threats. 

9  wet  and  5  dry  runs. 

29  Apr  98 

X 

Scheduled  by  range;  not  placed  on  AATC  schedule  because  AATC  deputy 
commander  for  operations  limited  JADS  support  to  one  mission  per  month 
during  April  and  May. 

7  May  98 

X 

Scheduled  range  time  was  taken  by  a  higher  priority  user. 

19  May  98 

X 

Scheduled  by  range;  not  placed  on  AATC  schedule  -  range  time  was  too 
early  (0900-1030)  for  AATC  to  support. 

20  May  98 

2  Hr. 

6th  and  final  risk  reduction  sortie.  1  F-16  (A/C  #1262)  versus  4  threats. 

14  wet  and  6  dry  runs. 

2.6.2  OAR  Baseline 

Fourteen  productive  range  hours  were  planned  to  be  flown  to  complete  the  OAR  baseline  test 
following  the  six  risk  reduction  missions  which  ended  in  May  1998.  In  actuality,  some  of  the  data 
collected  during  risk  reduction  missions  #5  and  #6  were  included  in  the  OAR  baseline  database 
because  the  test  profile,  ROE,  and  test  conditions  had  been  stabilized  sufficiently  to  make  these 
data  usable  for  record.  The  first  dedicated  OAR  baseline  mission  was  flown  on  8  June  1998. 
Some  of  the  same  scheduling/execution  problems  experienced  in  risk  reduction  continued  to 
plague  the  OAR  baseline  planning  and  some  missions  flown  were  marginally  productive.  All  of 
these  problems  together  resulted  in  a  total  of  17  OAR  baseline  missions  of  1.5  to  2.0  hours  of 
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range  time  being  scheduled  and  seven  missions  actually  being  flown.  The  total  number  of  range 
hours  (including  3.5  hours  from  the  last  two  risk  reduction  missions)  was  14.4  hours.  The  final 
OAR  baseline  mission  was  flown  on  20  August  1998. 

Table  1 1  summarizes  the  chronology  of  the  major  scheduled  events  during  this  portion  of  Phase  1 
testing. 


Table  11.  OAR  Baseline  Schedule  History 


Canceled 

Comments 

MMfogTM 

(Risk  reduction  MSN)  Some  data  valid  for  OAR  database.  10  wet,  4  dry  runs. 

MM— 

(Risk  reduction  MSN)  Some  data  valid  for  OAR  database.  10  wet,  4  dry  runs. 

2  Hr. 

21  runs  (16  wet,  5  dry).  Numerous  late  test  condition  calls  and  pilot 
problems. 

X 

Not  accepted  by  AATC  -  take  off  time  too  early  to  get  aircraft  support  at 
Tucson. 

16  Jun  98 

1.5  Hr. 

17  runs  (1 1  wet,  6  dry).  Some  problems  with  SADS  III  and  SADS  VI. 

17  Jun  98 

1.3  Hr. 

15  runs  (1 1  wet,  4  dry).  Aircraft  forced  to  climb  to  15K  feet  by  range  safety. 

19  Jun  98 

X 

Canceled  due  to  problem  at  range.  AATC  unable  to  replan  mission  in  time. 

23  Jun  98 

1.7  Hr. 

18  runs  (12  wet,  6  dry).  Some  problems  with  SADS  III  and  SADS  VI. 

ESAMJHI 

X 

Canceled  on  day  of  mission  due  to  SADS  III  magnetron  arching. 

EHfEEM 

1.1  Hr. 

Troubleshooting  MSN  to  isolate  a  spurious  signal.  Backup  ECM  pod  #0631. 

mu 

* 

*  MSN  given  to  AATC  for  their  ALR-69  software  test.  No  JADS  data 
collected. 

KIM 

1.8  Hr. 

20  runs  (17  wet,  3  dry).  Isolated  spurious  signal  associated  with  SADS  VI. 

17  Aug  98 

X 

Canceled  just  prior  to  take  off  at  Tucson  due  to  F-16  leading  edge  slat 
inoperable. 

20  Aug  98 

1.5  Hr. 

17  runs  (13  wet,  4  dry).  SADS  VI  inoperable  for  1st  9  runs,  then  slaved  to 
Target  Acquisition  and  Tracking  System  (TATS). 
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2.6.3  HITL 


Table  12  summarizes  the  HITL  event  history. 

Table  12.  HITL  Event  History 


Scheduled  Date 

Actual 

Completion 

Date 

Event 

Comments 

4  Jun  -  22  Jul  98 

23  Jul  98 

Prepare  Flight  Profile  Tapes  (AFEWES) 

MSN  5-10  completed 

19  Jul 

MSN  1 1  included  late, 

22  Jul  98 

17  Jul  98 

Test  Plan  Notes  (AFEWES) 

21  Jul  98 

ECM  Pod  Set  Up  (AFEWES) 

20  Jul  98 

20  Jul  98 

JETS  Set  Up  (AFEWES) 

20  -24  Jul  98 

19  Jul  98 

SADS  VIII  and  WEST  X  Set  Up  (AFEWES) 

27  -31  Jul  98 

31  Jul  98 

SADS  III  and  SADS  VI  Set  Up  (AFEWES) 

27  -30  Jul  98 

30  Jul  98 

Execute  HITL  Part  1  (AFEWES) 

3  -6  Aug  98 

7  Aug  98 

Execute  HITL  Part  2  (AFEWES) 

Lost  one  day  of  testing  for 
sim  problems 

15  Aug  98 

HITL  Quick-Look  Report 

10  Aug  -  Sep  98 

pending 

Data  Reduction  (AFEWES) 

Week  1  data  retrieved 
from  optical  disk 

Oct  -  Dec  98 

pending 

ADRS  MOP  Evaluation  (JADS) 

5  MOPs  not  coded 
correctly  in  ADRS 
software 
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3.0  Execution  Results 


3.1  Risk  Reduction  Execution 

The  risk  reduction  flights  had  many  objectives  to  accomplish  using  a  fly-fix-fly  approach  to  the 
missions.  The  following  items  were  of  major  interest  and  changed,  sometimes  significantly,  over 
the  period  the  risk  reduction  missions  were  flown. 

•  Flight  profile 

•  Threat  engagement  regions 

•  ECM  PT/MT  parameters 

•  Situational  awareness 

•  Test  control 

•  Instrumentation 

•  Data  collection  methods 

•  Jammer  performance 

•  Threat  performance 

•  Rules  of  engagement 

•  Analysis  process  development  and  execution 

•  Threat  system  orientation/performance  awareness 

3.1.1  Mission  #1,  28  Aug  97 

OVERVIEW:  Mission  #1  was  expected  to  yield  some  initial  data  to  begin  the  analysis  process 
execution,  familiarize  the  participants  with  the  OAR  test  control  facilities,  and  test  the  jammer  and 
threat  performance  and  interactions  using  the  flight  profile  and  engagement  regions  designed  at 
that  point. 

POSITIVE  RESULTS:  Once  the  test  team  arrived,  an  initial  familiarization  of  the  test  control 
facility  took  place.  The  test  team  became  acutely  aware  that  situational  awareness  (SA)  during 
the  test  was  of  paramount  importance  and  needed  extra  attention.  The  flight  profile  and 
engagement  regions  were  tested  but  needed  improvement  (see  below).  The  jammer  and  threat 
interactions  were  also  tested  and  showed  that  they  also  needed  extra  attention. 

ROOM  FOR  IMPROVEMENT:  Unfortunately,  the  mission  failed  to  meet  most  of  the  objectives. 
There  were  many  problems  just  getting  the  personnel  to  the  range  and  into  the  right  areas.  The 
jammer  instrumentation  was  not  working  as  expected  and  did  not  provide  any  data  to  evaluate  the 
jammer  performance.  While  the  mission  was  flown,  it  was  noted  that  the  WEST  X  had  trouble 
engaging  the  target,  and  the  racetrack  flight  profile  was  inadequate  for  the  purposes  of  the  EW 
Test.  Problems  were  found  in  the  flight  profile,  threat  system  orientation/performance  awareness, 
threat  engagement  regions,  ECM  PT/MT  parameters,  test  situational  awareness,  test  control,  data 
collection  instrumentation,  and  threat  performance. 
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3.1.2  Mission  #2, 18  Nov  97 


OVERVIEW:  While  waiting  several  months  for  another  mission  to  be  flown,  the  data  collection 
methods  were  modified  to  streamline  the  process  of  data  delivery  from  the  OAR.  Changes  were 
also  made  to  the  areas  mentioned  in  Mission  #1. 

POSITIVE  RESULTS:  The  mission  was  flown,  and  the  threat  and  jammer  interactions  improved 
and  produced  better  jammer  and  threat  performance  results.  The  implementation  of  site  observers 
took  place  during  this  mission  and  showed  a  remarkable  improvement  in  effective  test  situational 
awareness.  The  changes  in  test  control  also  resulted  in  marked  improvement  in  effectiveness,  and 
the  personnel  issues  of  getting  the  right  people  to  the  right  mission  locations  ran  much  more 
smoothly. 

ROOM  FOR  IMPROVEMENT:  While  the  mission  was  flown  with  the  improvements  mentioned 
above,  several  new  problems  to  be  worked  were  identified.  The  site  observers  did  not  receive  an 
adequate  orientation  to  the  mission  control  room,  which  caused  a  loss  of  situational  awareness 
during  the  first  several  runs  during  the  mission.  There  were  also  problems  in  following  the  rules 
of  engagement  because  the  operators  were  using  unauthorized  (not  in  accordance  with  the  JADS 
ROE)  modes  that  could  not  be  duplicated  in  our  ADS  environment,  and  the  threats  were  not 
engaging  the  target  at  the  specified  times  as  designated  by  the  mission  control.  There  were  also 
problems  in  the  instrumentation  performance  of  the  RAJPO  pods.  They  did  not  perform  well 
during  the  entire  mission.  Also,  one  of  the  two  ALQ-131  pods  was  not  performing  correctly  and 
needed  to  be  reset  during  the  mission  execution.  There  were  still  problems  noted  in  situational 
awareness,  test  control,  data  collection  instrumentation,  data  collection  methods,  jammer 
performance,  threat  performance,  and  rules  of  engagement. 

3.1.3  Mission  #3,  28  Jan  98 

OVERVIEW:  Because  of  the  holiday  schedule  and  the  delay  in  the  delivery  of  the  range  data  to 
JADS,  this  mission  took  place  10  weeks  after  the  previous  one.  The  ROE  had  been  modified,  as 
well  as  the  mission  tape  used  in  the  pod.  The  mission  was  more  successful  than  the  previous  two 
missions,  but  there  were  still  many  problems  to  be  worked. 

POSITIVE  RESULTS:  The  data  collected  from  the  mission  were  more  accurate  and  reliable  than 
on  previous  missions,  and  the  new  and  improved  observer  assignments  were  handled  more 
expertly  to  provide  better  test  control  and  situational  awareness.  The  aircraft,  jammer  pod,  and 
Firefly  instrumentation  seemed  to  have  minimal  problems,  but  new  problems  surfaced  with  regard 
to  aircraft  profile,  ALQ-13 1  recorder  card  accuracy,  and  threat  engagement  regions. 

ROOM  FOR  IMPROVEMENT:  While  ALQ-131  pod  data  were  collected,  the  issue  of  the 
accuracy  of  the  recorder  card  clock  came  under  serious  scrutiny.  The  ALQ-131  recorder  card 
showed  a  drift  in  the  timing  clock  of  several  seconds,  which  invalidated  the  timing  MOP  data 
gathered  from  the  mission.  The  aircraft  profile  caused  a  lot  of  problems  for  the  threats  engaging 
the  target  because  of  ground  clutter.  What  resulted  was  a  series  of  breaklocks  and  poor  tracking 
that  was  not  due  to  the  jammer  pod,  but  from  ground  clutter  instead.  The  mission  tape  flown  in 
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this  mission  also  showed  the  jammer  tracking  other  radar  emitters  which  were  not  programmed 
for  the  EW  Test.  This  may  have  impacted  the  workload  on  the  jammer  pod.  As  a  result,  it  was 
decided  to  use  a  specialized  mission  tape  created  solely  for  the  JADS  EW  Test.  RDAPAS 
instrumentation  had  failed  at  the  SADS  III,  which  prevented  the  collection  of  J/S  data  for  the  site, 
and  SASY  instrumentation  failed  causing  a  decrease  in  situational  awareness  of  the  RF 
environment.  The  final  analysis  of  the  range  data  provided  a  first  look  into  a  full  mission  data 
analysis  process,  where  lack  of  familiarity  with  the  data  reduction  software,  ADRS,  was  a  big 
problem.  It  was  evident  that  more  training  on  the  use  of  ADRS  was  required,  and  GTRI  provided 
on-site  support  to  help  resolve  this  problem.  The  threat  observers,  while  now  more  familiar  with 
the  mission  control  center  (MCC)  were  not  able  to  detect  when  a  threat  was  or  was  not 
performing  as  prescribed  by  the  ROE,  which  also  affected  the  results  of  the  mission  data. 
Problems  were  noted  in  the  areas  of  flight  profile,  threat  engagement  regions,  ECM  MT 
parameters,  analysis  procedure  development  and  execution,  and  threat  system  awareness. 

3.1.4  Mission  #4,  25  Feb  98 

OVERVIEW:  Four  weeks  had  passed  since  the  previous  mission,  so,  once  again,  the  operators 
were  not  familiar  with  the  ROE.  ROE  had  been  revised,  the  observers  had  become  more  aware  of 
the  threat  system  operations,  the  flight  profile  and  threat  engagement  regions  had  been  modified, 
and  a  new  mission  tape  had  been  loaded  in  the  jammer  pod.  All  of  these  changes  demonstrated 
that  JADS  intervened  to  solve  previous  problems,  and  once  again,  gave  opportunity  for  new 
problems  and  issues  to  arise. 

POSITIVE  RESULTS:  The  JADS  team  held  a  meeting  with  the  threat  operators  prior  to  the 
mission  to  discuss  the  peculiarities  of  each  threat  system  and  how  each  system  engaged  the  target. 
This  meeting  greatly  improved  observer  awareness  of  the  threat  system,  but  still  left  some 
ambiguity  as  evidenced  from  some  operator  comments.  The  agreement  at  this  meeting  was  made 
for  the  operators  to  actually  host  the  observers  at  the  threat  site  to  improve  situational  awareness. 
The  new  mission  tape  decreased  the  opportunity  for  extraneous  emitters  to  affect  the  test 
execution.  The  flight  profile  was  raised  to  9000  MSL,  and  coupled  with  the  new  threat 
engagement  regions,  the  ability  for  the  threats  to  engage  the  target  were  maximized  compared  to 
all  previous  missions.  A  new  procedure  had  been  developed  at  AATC  to  improve  the  timing 
accuracy  of  the  ALQ-131  recorder  cards. 

ROOM  FOR  IMPROVEMENT:  While  this  mission  was  better  executed  from  a  mission  control 
standpoint  and  the  best  results  were  collected  compared  to  any  previous  missions,  there  were  still 
problems  to  be  worked  in  many  areas.  The  mission  tape  parameters  had  not  been  finalized  against 
the  WEST  X,  which  made  the  data  unusable  for  the  OAR  baseline  data  population.  The  ALQ- 
131  recorder  cards  still  had  a  timing  accuracy  problem.  This  problem  resulted  in  the  cards  being 
sent  to  the  manufacturer  for  check-out.  The  Firefly  instrumentation  had  another  failure  in  the 
1553  card,  which  resulted  in  no  data  being  gathered  on  the  jammer  pod  from  the  Firefly.  This 
second  failure  of  the  1553  card  coupled  with  other  tests  and  research  performed,  caused  the  1553 
card  to  be  dropped  from  the  instrumentation  configuration  altogether.  It  appeared  that  not  only 
did  the  1553  card  have  sporadic  performance,  it  also  affected  the  performance  of  the  jammer  pod. 
However,  the  complete  analysis  process  was  executed  with  GTRI  support  and  resulted  in  more 
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improvements  made  to  the  ADRS.  The  RDAPAS  instrumentation  failed  on  the  SADS  VIII 
system,  which  caused  a  total  loss  of  J/S  data  on  the  SADS  VIII  for  the  mission.  Problems  were 
also  noted  in  the  areas  of  ECM  mission  tape  parameters,  analysis  process  execution, 
instrumentation,  data  collection  methods,  rules  of  engagement,  and  threat  system  awareness. 

3.1.5  Mission  #5,  28  Apr  98 

OVERVIEW:  This  was  the  first  mission  counted  toward  the  OAR  data  population.  The 
reference  test  condition  (RTC)  had  been  finalized  with  respect  to  flight  profile,  threat  engagement 
regions,  ECM  PT  and  MT  parameters,  instrumentation,  and  rules  of  engagement.  The  finalization 
of  these  RTC  areas  resulted  in  an  orderly  mission  execution.  There  were  also  improvements  in 
the  data  collection  process. 

POSITIVE  RESULTS:  This  was  the  first  mission  where  the  observers  performed  their  job  on¬ 
site  at  the  threat  site  (except  the  WEST  X).  This  improved  situational  and  threat  performance 
awareness  and  allowed  for  a  great  deal  of  observer/operator  interactions  for  the  JADS  team 
members  to  become  aware  of  the  operations  of  the  threat  systems.  While  this  mission  was  initially 
reported  as  marginally  productive,  changes  in  the  interpretation  of  the  SADS  VI  data  revealed  this 
mission  to  be  more  useful  than  originally  expected.  The  modified  ROE  allowed  the  WEST  X  to 
have  a  better  engagement  with  the  target  on  the  northbound  runs.  This  proved  the  flight  profile 
and  threat  engagement  regions  were  adequate  enough  to  provide  reasonable  and  repeatable  data. 
The  interactions  between  the  WEST  X,  SADS  VIII  and  jammer  were  clearly  seen  in  the  results. 
This  proved  the  PT  and  MT  parameters  were  performing  as  designed  and  finalized.  The 
elimination  of  the  Southwest  Research  1553  card  allowed  for  more  consistent  jammer 
performance  against  the  threats. 

ROOM  FOR  IMPROVEMENT:  While  the  observers  and  operators  had  more  interaction,  the 
new  environment  proved  disturbing  to  some  observers.  The  confusion  resulted  in  improper 
engagements  for  the  SADS  VI  for  most  of  the  mission.  This  necessitated  an  earlier  arrival  time 
for  observers  at  the  threat  sites  in  future  missions  to  coordinate  the  ROE.  It  was  also  noted  that 
observer  communications  to  the  MCC  could  be  improved  to  allow  for  a  better  real-time 
assessment  of  the  mission  execution.  The  SADS  III  had  a  communications  failure  that  did  not 
allow  it  to  engage  the  target  for  8  of  the  14  runs  in  the  mission.  This  was  a  hardware  problem 
that  was  resolved  by  the  OAR  personnel  prior  to  the  next  mission.  The  RDAPAS  systems  failed 
at  least  once  for  each  instrumented  threat  during  the  mission.  The  pilot  used  a  noninstrumented 
switch  to  inhibit  pod  radiation  for  dry  runs,  which  caused  confusion  in  the  data  reduction  results. 
This  mode  had  been  deleted  previously,  and  it  was  reiterated  to  the  pilot  to  turn  the  pod  to 
STANDBY  mode,  not  INHIBIT  mode.  Problems  noted  from  this  mission  were  in  the  areas  of 
situational  awareness,  test  control,  and  data  collection  methods. 
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3.1.6  Mission  #6,  20  May  1998 


OVERVIEW:  This  was  the  second  mission  where  data  were  used  in  the  OAR  baseline 
population.  With  the  changes  made  in  the  areas  noted  from  the  previous  mission,  this  was  the 
most  successful  mission  to  date. 

POSITIVE  RESULTS:  The  observer  team  had  adjusted  their  schedule  to  arrive  at  the  threat  site 
early  enough  to  brief  the  operators  on  the  ROE  and  answer  any  questions  about  the  day’s  mission. 
This  resulted  in  better  compliance  by  all  threats  from  the  ROE  standpoint.  The  OAR  personnel 
also  set  up  an  observer  net  specifically  for  the  observers  to  communicate  their  issues  to  the  MCC. 
This  setup  allowed  the  head  observer  in  the  MCC  to  keep  in  touch  with  the  observers  during  the 
mission  and  resulted  in  much  better  test  control  and  situational  awareness  for  both  the  observers 
at  the  threat  sites  and  the  test  controllers  in  the  MCC.  All  threats  had  valid  engagements  with  the 
target.  The  full  analysis  process  was  exercised  and  sample  MOP  data  were  gathered  for  all 
MOPs  except  the  timing  MOPs  (correct  threat  identification  (ID)  response  time  and  correct  ECM 
selection  technique  response  time).  The  analysis  process  had  also  been  improved  to  include 
screening  criteria  for  runs  that  met  the  reference  test  condition.  With  these  improvements,  the 
data  used  for  the  OAR  baseline  population  were  collected  and  analyzed  according  to  the 
applicable  part  of  the  correlation  process  procedures.  The  missions  conducted  after  mission  #6 
were  all  slightly  improved  as  the  operators,  observers,  and  test  control  team  became  more  aware 
of  threat  operations  and  were  able  to  identify  system  anomalies  in  real-time  vice  post-test  analysis. 

ROOM  FOR  IMPROVEMENT:  The  SADS  III  had  a  system  failure  and  was  down  for  the  first 
six  runs  of  the  mission.  The  problem  was  solved  after  run  6,  but  the  threat  still  had  trouble 
acquiring  the  target  on  some  runs.  There  was  still  room  for  improvement  with  the  SADS  VI 
operations.  The  threat  was  having  trouble  maintaining  a  track  on  the  target  on  4  of  the  20  runs. 
The  ROE  were  reiterated  to  the  threat  operators  to  help  them  understand  that  the  EW  Test  was 
structured  to  minimize  the  operator  interactions  with  the  target  and  to  maximize  the  interactions 
between  the  threat  and  the  jammer  pod.  The  jammer  performance  against  the  SADS  III  looked 
irregular  because  SASY  showed  intermittent  jamming  throughout  the  day.  The  jammer  performed 
well  against  the  other  threats.  This  problem  was  carried  through  to  the  next  mission  and  was  later 
addressed  after  mission  #9.  There  was  also  a  problem  with  extraneous  emitters  radiating  during 
the  test  execution.  While  the  emissions  of  an  unplanned  emitter  did  not  affect  the  jammer,  they 
did  interfere  with  the  acquisition  radar  of  the  SADS  VIII  causing  some  engagements  to  be  shorter 
than  others.  This  problem  was  rectified  by  the  program  manager  at  the  OAR,  and  the  unplanned 
emitter  was  shut  down  after  10  minutes  of  radiating.  This  was  the  final  risk  reduction  mission, 
and  the  problems  encountered  seemed  to  be  more  due  to  the  procedures  and  policies  at  the  OAR 
than  in  the  processes  controlled  by  JADS.  The  improvements  after  this  mission  were  changes  in 
the  analysis  process,  test  control,  and  situational  awareness.  The  RTC  did  not  change  after  this 
mission,  and  all  successive  missions  were  counted  toward  the  OAR  baseline  data  population. 
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3.2  OAR  Baseline  Execution 


The  basic  objective  of  the  OAR  baseline  missions  was  to  collect  baseline  ALQ-131  effectiveness 
data  to  answer  the  MOPs  (listed  in  Table  1)  for  comparison  with  results  that  will  be  obtained  from 
Phases  2  and  3.  These  data  were  collected  during  missions  5  through  13  of  the  OAR  baseline 
phase.  The  following  table  provides  a  summary  of  the  mission  productivity  during  the  OAR 
baseline  testing.  This  table  shows  the  total  number  of  runs  attempted  during  each  mission,  and 
the  subset  of  usable  data  runs  achieved  by  each  threat.  The  difference  was  due  to  threat 
equipment  failures,  invalid  reference  conditions,  and  runs  with  no  data  recorded.  Risk  reduction 
missions  #5  and  #6  are  included  because  OAR  baseline  data  for  record  were  taken  from  portions 
of  these  missions. 


Table  13.  OAR  Baseline  Mission  Summary 

(Number  of  Usable  Data  Runs  by  Threat) 


Threat 

Mission 

SADS  III 

SADS  VI 

SADS  VIII 

WEST  X 

Average 
Success  Rate 

Mission  (#  total  runs) 

Number  of  usable  data  runs 

Mission  5 
(14  runs) 

6 

12 

13 

13 

79% 

Mission  6 
(20  runs) 

13 

16 

20 

15 

80% 

Mission  7 
(21  runs) 

19 

16 

20 

19 

87% 

Mission  8 
(17  runs) 

15 

8 

17 

17 

84% 

Mission  9 
(15  runs) 

4 

4 

4 

4 

27% 

Mission  10 
(18  runs) 

18 

17 

18 

99% 

Mission  11 
(13  runs) 

11 

11 

12 

10 

85% 

Mission  12 
(20  Runs) 

14 

14 

18 

20 

83% 

Mission  13 
(17  Runs) 

16 

0 

17 

17 

74% 

#  total  runs 
(155  attempted) 

116 

99 

138 

133 

78% 

There  are  four  items  to  highlight  from  this  table.  The  overall  mission  productivity  was  about 
78%,  but  the  reliability  of  individual  threat  simulators  varied  quite  a  bit.  This  was  mostly  due  to 
random  mechanical  failures  at  the  threat  sites  on  different  days,  lightning/weather-related  down 
times  at  individual  sites,  and  occasionally  because  of  communication  failures  from  the  control 
center.  (Examples:  the  missile  simulation  system  crashed  at  one  site  for  6  runs;  the 
communications  link  was  down  from  MCC  to  one  site  for  6  runs;  a  lightning  storm  near  one  site 
caused  it  to  shut  down  for  6  runs;  a  recorder  broke  at  one  site  for  the  last  9  runs)  These  types  of 
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equipment  failures  and  weather  delays  automatically  caused  a  run  to  be  bad  because  no  data  were 
produced. 

The  next  notable  item  was  the  low  production  on  Mission  #9.  Only  4  of  the  1 5  runs  were  usable, 
because  11  of  the  profiles  varied  from  the  reference  test  condition.  The  test  range  had  an 
explosives  project  underway  at  the  south  end  of  the  JADS  profile  which  required  the  pilot  to 
climb  from  9000  to  1 5000  feet  during  the  last  1 1  runs.  This  climb  and  descent  during  the  runs 
significantly  changed  the  apparent  radar  cross-section  and  therefore  changed  the  radar  tracking 
performance.  Thus,  1 1  runs  were  deemed  invalid  on  Mission  #9. 

Another  notable  item  was  related  to  the  SADS  VI.  On  Mission  #13,  the  target  tracking  radar  had 
a  mechanical  failure  and  was  not  able  to  track  the  aircraft  on  its  own  that  day.  The  site  was  only 
able  to  slave  its  antenna  to  an  external  source  but  still  did  not  provide  valid  data.  Therefore,  none 
of  the  17  runs  from  SADS  VI  were  usable  on  Mission  #13,  which  lowered  its  overall  productivity 
to  63%  during  the  OAR  baseline  phase.  (If  those  runs  had  been  good,  the  SADS  VI  would  have 
had  a  74%  productivity  rating.) 

The  next  section  will  discuss  the  OAR  baseline  data  collection  and  results  for  each  of  the  ten 
measures  of  performance  that  were  described  in  Table  1.  The  JADS  EW  Test  team  had  the 
function  of  characterizing  all  of  the  data  collected,  while  GTRI  provided  all  of  the  statistical 
analysis  for  MOP  results.  For  each  OAR  threat  system,  there  are  four  run  condition  categories 
for  which  MOPs  will  be  evaluated:  northbound  dry  (ND),  southbound  dry  (SD),  northbound  wet 
(NW),  and  southbound  wet  (SW).  However,  not  all  MOPs  can  be  calculated  for  every  summary 
condition  or  for  every  threat.  Table  14  shows  the  MOPs  collected  for  each  threat  and  condition. 

Table  14.  EW  Test  Measures  of  Performance 


MOP 

Conditions 

SADS  III 

SADS  VI 

SADS  VIII 

WEST 

X 

Correct  threat  ID 

NW,SW 

X 

X 

X 

X 

Correct  threat  ID  response  time* 

NW,SW 

Correct  ECM  technique  selection 

NW,SW 

X 

X 

X 

X 

Correct  ECM  technique  selection 
response  time* 

NW,SW 

Jamming-to-signal  ratio  versus 
threats** 

NW,SW 

Ml 

Increase  in  tracking  error  versus  threat 

ND+NW 

SD+SW 

X 

*** 

X 

Number  of  breaklocks 

ND,SD 

NW,SW 

X 

*** 

X 

X 

Reduction  in  engagement  time 

ND+NW 

SD+SW 

X 

*** 

X 

X 

Reduction  in  missiles  launched 

ND+NW 

SD+SW 

X 

X 

X 

Missile  miss  distance 

ND,SD 

NW,SW 

X 

X 

X 
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•  These  response  time  MOPs  were  not  computed  in  the  OAR  baseline  test  because  of  inaccurate 
timing  instrumentation  on  the  jamming  pod.  The  missing  data  were  later  obtained  via  the  SIL  test 
at  Eglin  AFB  in  September  1998. 

**  The  jamming-to-signal  ratio  data  were  not  collected  in  the  OAR  baseline  test  data  because  of 
problems  with  instrumentation. 

***  These  MOPs  for  the  SADS  VI  were  not  computed  because  there  is  no  SADS  VI  target 
tracking  radar  (TTR)  implementation  at  AFEWES  to  compare  to  the  OAR  baseline  data. 

3.2.1  Correct  Threat  ID 

The  correct  threat  ID  MOP  measures  the  accuracy  of  the  jammer’s  ability  to  correctly  identify  a 
threat  system  as  the  threat  begins  radiating  the  target.  Only  wet  run  data  were  analyzed  for  this 
MOP.  Three  conditions  had  to  be  met  for  the  jammer  to  provide  a  threat  ID. 

•  The  threat  must  be  radiating  in  a  mode  that  the  jammer  is  designed  to  detect, 

•  The  threat  composite  angular  tracking  error  must  be  below  a  user-defined  threshold  (for  the 
JADS  EW  Test  this  threshold  was  half  a  beam  width  for  each  threat)  for  a  time  greater  than 
the  expected  response  time,  and 

•  The  threat  must  be  within  the  jammer  detection  range  for  a  time  greater  than  the  expected 
response  time.  Since  the  JADS  RTC  assumes  condition  three  will  be  met  at  all  times,  the 
detection  range  threshold  is  set  to  infinite  range. 

Each  usable  wet  run  provided  at  least  one  chance  for  a  correct  threat  ID.  The  sample  size  for  this 
MOP  was  at  least  the  number  of  usable  data  runs  from  each  threat  when  the  ALQ-131  was  on. 
There  were  adequate  samples  for  correct  threat  ID. 

3.2.2  Correct  Threat  ID  Response  Time 

This  MOP  was  designed  to  measure  each  time  a  correct  threat  ID  was  recorded  in  the  receiver 
processor  of  the  jammer.  Specifically,  this  MOP  measured  the  time  taken  to  correctly  identify  a 
threat  from  the  time  when  a  threat  radiated  and  was  satisfactorily  tracking  the  target  to  when  the 
receiver  processor  reported  the  correct  ID. 

This  MOP  could  not  be  evaluated  during  the  OAR  baseline  because  of  inaccurate  timing 
instrumentation  on  the  jamming  pod. 

3.2.3  Correct  ECM  Technique  Selection 

A  correct  ECM  technique  selection  was  defined  as  the  selection  of  the  correctly  programmed 
ECM  technique  against  a  threat.  To  assess  correct  ECM  technique  selection  accuracy,  the  only 
condition  that  must  be  met  for  the  jammer  and  RTC  was  that  a  correct  threat  ID  must  exist  for  an 
amount  of  time  greater  than  the  minimum  ECM  response  time.  If  the  jammer  correctly  identifies 
the  threat,  ADRS  checks  to  see  if  the  ECM  technique  selected  matches  the  expected  technique 
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number  in  the  emitter  descriptor  file.  If  the  jammer  does  not  correctly  ID  the  threat,  an  incorrect 
ECM  technique  selection  is  not  counted  against  the  jammer’s  performance. 

3.2.4  Correct  ECM  Technique  Selection  Response  Time 

A  correct  ECM  technique  response  time  can  only  be  measured  when  a  correct  ECM  technique 
selection  has  occurred.  To  assess  correct  ECM  technique  selection  accuracy  response  time,  two 
conditions  had  to  be  met. 

•  A  correct  ECM  technique  must  be  selected,  and 

•  The  threat  composite  angle  tracking  error  must  be  no  more  than  a  half  beam  width  from  the 
target  for  an  interval  of  time  greater  than  the  expected  ECM  response  time. 

This  MOP  could  not  be  evaluated  during  the  OAR  baseline  because  of  inaccurate  timing 
instrumentation  on  the  jamming  pod. 

3.2.5  Jamming-to-Signal  Ratio  Versus  Threats 

Jamming-to-signal  ratio  (J/S)  is  defined  as  the  measure  of  jamming  signal  strength  divided  by 
threat  signal  strength.  The  goal  of  this  MOP  was  quantification  of  the  J/S  over  the  engagement. 
J/S  was  collected  as  time-series  or  type  3  data  and  converted  to  type  1  data  for  analysis. 

The  J/S  data  were  not  able  to  be  collected  in  the  OAR  baseline  test.  The  test  range  RDAP AS 
instrumentation  had  been  planned  to  be  used  for  extracting  J/S  data.  RDAP  AS  had  never  been 
used  for  this  application,  however,  it  is  similar  to  instrumentation  at  the  Eglin  range  that  is  used 
for  this  purpose.  GTRI  designed  the  Eglin  instrumentation  and  agreed  to  attempt  to  extract  J/S 
from  RDAPAS  data.  Data  formatting  problems  had  to  be  resolved  before  J/S  could  be  extracted. 
These  problems  consumed  several  months  and  the  entire  budget  for  this  task.  More  funds  were 
allocated,  the  formatting  problems  were  resolved,  and  software  was  created.  The  results 
produced  erratic  J/S  from  the  RDAPAS  data.  Investigation  by  GTRI  and  the  WTR  revealed  that 
the  RDAPAS  instrumentation  was  being  operated  in  a  nonlinear  area  of  its  response  curve.  This 
precluded  accurate  (within  2  decibel  [dB])  calibration.  Rather  than  redesign  the  RDAPAS,  JADS 
elected  to  drop  J/S  as  a  MOP  from  the  OAR  baseline  test. 

3.2.6  Increase  in  Tracking  Error  Versus  Threat 

This  MOP  involved  a  comparison  of  tracking  error  against  a  target  between  wet  and  dry  runs  with 
the  goal  of  quantifying  the  percentage  difference  between  wet  runs  and  an  established  dry  run 
baseline.  Tracking  error  data  were  collected  as  time-series  or  type  3  data  and  converted  to  type  1 
data  for  analysis.  The  tracking  error  values  were  taken  directly  from  the  target  tracked  position 
file  delivered  from  the  OAR.  To  collect  tracking  error  data,  a  good  track  must  be  established. 
Good  tracks  are  established  when: 
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•  The  tracking  error  is  below  the  threshold, 

•  The  threat  is  operating  in  a  good  track  mode,  and 

•  The  operator  maintains  that  good  track  for  a  continuous  two  seconds 

3.2.7  Number  of  Breaklocks 

This  MOP  produces  type  1  data.  The  goal  of  this  MOP  was  quantification  of  the  number  of 
breaklocks  per  run  for  both  wet  and  dry  engagements. 

In  order  for  a  breaklock  to  occur,  the  threat  must  transition  from  a  good  track  mode  to  a  bad 
track  mode  after  the  first  good  track  is  established.  Two  conditions  must  be  met  for  the 
establishment  of  a  good  track. 

•  The  tracking  error  must  be  beneath  the  tracking  error  threshold  for  a  user-definable  time 
period  (usually  2  seconds),  and 

•  The  threat  must  switch  into  its  good  track  mode  (AUTO  mode). 

Following  the  establishment  of  a  good  track,  a  breaklock  occurs  when  either  or  both  of  the 
following  conditions  are  met. 

•  The  tracking  error  exceeds  the  threshold  for  a  user-definable  time  period,  or 

•  The  threat  must  switch  from  its  good  track  mode  into  another  radiating  mode. 

A  breaklock  does  not  occur  at  the  end  of  the  run  when  the  threat  transitions  to  “off’  mode. 

3.2.8  Reduction  in  Engagement  Time 

This  MOP  involves  a  comparison  of  good  engagement  time  between  wet  and  dry  runs  with  the 
goal  of  quantifying  the  percentage  difference  between  wet  runs  and  an  established  dry  run 
baseline.  Percentage  of  good  engagement  time  is  defined  as  the  percentage  of  time  a  threat  is  in  a 
good  track  mode  compared  to  the  total  engagement  time.  Good  and  total  engagement  time 
definitions  vary  for  specific  threat  systems. 

Total  engagement  time  for  the  SADS  III  and  SADS  VIII  is  the  total  time  the  threat  is  radiating  in 
a  mode  which  is  expected  to  be  jammed  by  the  ALQ-131.  Good  engagement  time  for  these  threat 
systems  is  the  total  time  a  threat  is  in  AUTO  mode  and  tracking  error  is  below  the  threshold  for  at 
least  a  user-definable  time  period.  AUTO  mode  is  defined  as  the  threat  system  radiating  with  both 
auto  angle  and  auto  range  track  loops  engaged.  All  other  radiating  modes  are  not  good  track 
modes. 

For  the  SADS  VI,  engagement  time  can  not  be  calculated  because  there  is  no  good  track  mode 
designation  for  the  missile  seeker  modes. 

Total  engagement  time  for  the  WEST  X  is  defined  as  the  total  time  the  threat  is  activated. 
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NOTE:  Since  the  AFEWES  implementation  of  the  WEST  X  does  not  deactivate  at  the  end  of 
the  run,  ADRS  must  adjust  the  total  engagement  time  to  eliminate  the  last  segment  of  the  TO 
ECM  OFF  mode  when  seen  in  the  AFEWES  data.  Good  engagement  time  for  the  WEST  X  is 
defined  as  the  time  when  the  threat  is  radiating  in  AUTO  mode  and  the  tracking  error  is  below  the 
threshold  for  at  least  a  user-definable  time  period. 

3.2.9  Reduction  in  Missiles  Launched 

This  MOP  involves  a  comparison  of  the  number  of  missiles  launched  between  wet  and  dry  runs 
with  the  goal  of  quantifying  the  percentage  difference  between  wet  runs  and  an  established  dry 
run  baseline.  The  missile  shot  data  are  delivered  from  the  OAR,  and  each  recorded  missile  event 
is  counted  as  a  missile  shot. 

3.2.10  Missile  Miss  Distance 

The  missile  miss  distance  MOP  was  calculated  with  the  intent  to  characterize  the  threat's 
performance  during  both  wet  and  dry  runs.  The  missile  shot  data  were  delivered  from  the  OAR 
for  the  SADS  III  and  SADS  VIII,  and  from  AFEWES  for  the  SADS  VI,  which  included  a  single 
number  for  distance  of  closest  approach  to  the  aircraft. 

3.3  HITL  Execution 

3.3.1  Day  #1,  27  Jul  98 

OVERVIEW:  Day  one  of  testing  began  with  the  SADS  VIII  and  WEST  X  manned  simulators 
while  JETS  replicated  the  RF  signals  of  the  SADS  VI  and  SADS  III.  The  data  post-processing 
tool — ADRS — was  not  fully  exercised  because  the  initial  log  files  sent  to  GTRI  were  missing 
critical  threat  performance  data  and  had  incorrect  aircraft  flight  data.  JADS  also  determined  that 
AFEWES  only  had  the  input  function  of  the  logger  input  and  output  recording  capabilities 
collecting  data.  The  AFEWES  software  published  the  aircraft  script  and  subscribed  to  the  threat 
performance  data.  The  logger  was  supposed  to  record  both  sets  of  data.  AFEWES  updated  the 
logger  software,  and  the  testing  began.  AFEWES  was  able  to  record  the  threat  simulator  activity 
mode  changes  using  their  proprietary  optical  disk  software  in  parallel  with  the  RTI  logger 
software.  Because  of  our  compressed  preparation  and  execution  schedule,  AFEWES  normal  data 
collection  method  (optical  disk)  was  our  reliable  backup  system. 

RESULTS:  JADS  executed  31  total  runs  and  25  were  considered  usable  for  analysis.  As  for  the 
federate,  there  were  other  critical  parameters  missing  from  the  logger  files,  but  AFEWES  changed 
their  federate  code,  and  all  observable  threat  parameters  were  collected.  Additionally,  all  data 
were  later  post-processed  from  the  optical  disk  software  and  used  for  MOP  analysis. 
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3.3.2  Day  #2,  28  Jul  98 


OVERVIEW:  The  HITL  testing  resumed  data  collection  with  the  SADS  VIII  and  WEST  X 
manned  simulators  while  the  JETS  replicated  the  RF  signals  for  the  SADS  VI  and  SADS  III.  Data 
from  OAR  baseline  mission  1 1  were  processed  by  the  Western  Test  Range  and  mailed  to  JADS 
earlier  than  expected.  AFEWES  was  able  to  produce  the  aircraft  scripts,  and  they  were  added  to 
our  HITL  test  matrix  which  bumped  the  run  count  up  to  62.  As  for  the  simulators,  the  initial 
condition  for  the  WEST  X  prior  to  tracking  the  target  was  about  180  degrees  from  the  direction 
of  the  target  aircraft — the  simulator  had  problems  acquiring  and  engaging  the  aircraft.  The  flight 
paths  from  south  to  north  changed  slightly  for  OAR  baseline  mission  #7  which  placed  the  aircraft 
reasonably  above  the  threat  and  at  an  elevation  and  azimuth  that  caused  the  simulation  to  crash. 
These  profiles  contributed  to  a  high  number  of  breaklocks  during  the  engagement  too.  Therefore, 
the  flight  profile  was  changed  to  accommodate  the  direction  of  these  runs — problem  solved. 
Another  major  item  noted  was  the  inability  to  time  synchronize  the  data  sources.  The  TMCs  had 
an  IRIG-B  capability,  but  it  was  not  operational.  The  time  displayed  within  the  TMC  was  time 
from  zero.  Once  the  TMC  initialized  and  the  runs  began,  the  TAMS  sent  a  start  command  to 
each  TMC  and  the  TMCs  began  recording.  Each  video  display  as  well  as  the  digibus  monitor  and 
logger  files  had  different  times.  (The  digibus  and  logger  files  received  time  from  an  IRIG-B 
source.)  AFEWES  corrected  this  problem  by  splitting  and  superimposing  GPS  time  onto  the 
videos  in  the  TMC.  However,  the  runs  from  day  one  did  not  have  a  suitable  time  source 
displayed  on  the  videos  for  troubleshooting  data  anomalies. 

RESULTS:  JADS  executed  51  total  runs  and  44  were  considered  usable  for  analysis.  To  assist 
with  the  problems  associated  with  missing  data,  JADS  sent  AFEWES  the  newly  created  logger 
reader  code  that  segmented  the  data  into  a  format  for  ADRS  and  also  displayed  the  data  in  a  text 
format.  This  logger  software  gave  AFEWES  and  the  JADS  observers  the  ability  to  view  threat 
performance  data.  At  this  point  we  were  only  able  to  verify  a  limited  portion  of  collected  logger 
data. 

3.3.3  Day  #3,  29  Jul  98 

OVERVIEW:  The  HITL  data  collection  continued  with  the  SADS  VIII  and  WEST  X  manned 
simulators  and  JETS  replicated  the  RF  signals  for  the  SADS  VI  and  SADS  III.  This  was  the  first 
day  where  JADS  observed  problems  with  the  SADS  VIII.  During  several  runs,  AFEWES 
experienced  intermittent  tracking  error  spikes.  After  exhaustive  investigation  AFEWES 
determined  the  problem  involved  the  method  used  to  compute  the  aircraft’s  velocity.  AFEWES 
had  used  the  current  and  previous  aircraft  positions.  This  method  added  an  unnecessary  amount 
of  jitter  that  caused  the  SADS  VIII  tracking  error  to  increase.  The  problem  was  avoided  by  using 
the  current  and  future  positions  in  the  aircraft  velocity  calculation.  This  solution  eliminated  the 
“lag”  and  resulted  in  smoother  target  data  inputs  for  both  the  SADS  VIII  and  WEST  X. 

RESULTS:  JADS  executed  40  total  runs  and  27  were  considered  usable  for  analysis. 
Additionally,  ADS  security  agreements  between  AFEWES  and  JADS  were  signed,  and  the  T-l 
hardware  was  installed  the  following  day.  The  T-l  was  installed  in  preparation  for  Phase  2,  which 


56 


was  to  begin  integration  one  month  after  the  HITL  test,  but  it  proved  to  be  a  convenient  method 
to  transfer  recorded  data  to  JADS. 

3.3.4  Day  #4, 30  Jul  98 

OVERVIEW :  This  was  the  final  day  of  testing  with  the  active  SADS  VIII  and  WEST  X  threats. 
During  each  run,  the  SADS  VIII  would  normally  fire  several  missiles.  The  TAMS  two- 
dimensional  simulation  display  depicted  the  appropriate  number  of  missile  firings.  The  raw  missile 
data  recorded  by  the  logger  and  viewed  on  one  of  the  test  control  room  monitors  displayed  data 
for  less  than  the  total  missiles  fired.  Also,  the  missile  data  displayed  had  constant  latitude  and 
longitude  values  (values  did  not  change)  and  all  zeros  for  the  altitude.  Although  this  situation 
seemed  unacceptable,  the  only  relevant  data  for  the  MOP  computation  were  the  missile  miss 
distances.  There  were  no  additional  changes  implemented  to  fix  this  problem. 

RESULTS:  Of  the  42  runs  executed,  40  were  good.  JADS  exceeded  our  goal  of  100  runs  by  32. 
There  were  a  total  of  62  scripts  generated  for  this  test,  and  we  successfully  completed — scored  as 
good — 132  runs.  Each  script  was  executed  at  least  twice. 

3.3.5  Day  #5, 3  Aug  98 

OVERVIEW:  AFEWES  operated  the  SADS  III  and  SADS  VI  while  the  JETS  replicated  the  RF 
signals  of  the  WEST  X  and  SADS  VIII.  This  week  began  with  several  unrecoverable  problems 
on  the  fifth  day  of  testing.  The  SADS  III  Honeywell  computer  had  a  core  memory  failure.  The 
component  was  replaced,  but  JADS  lost  most  of  the  testing  day.  AFEWES  also  encountered 
several  problems  with  the  SADS  VI.  Initially,  the  SADS  VI  did  not  accept  the  aircraft 
information  generated  from  TAMS.  After  rectifying  this  problem,  one  additional  problem  was 
found  during  the  diagnosis  of  the  initial  problem.  The  location  of  the  “true  target”  versus  the 
SADS  VI  “perceived  target” — the  differences — equated  to  tracking  error.  This  situation 
prevented  the  threat  simulator  from  adequately  engaging  the  target.  AFEWES  modified  the 
simulator  to  accept  the  true  target  position  which  assumed  perceived  location  equaled  true 
location.  We  observed  no  further  engagement  anomalies. 

RESULTS:  Although  no  test  runs  were  conducted,  AFEWES  corrected  potential  sources  of 
error  that  could  have  negatively  influenced  the  data  used  in  calculating  the  MOPs.  There  were  no 
other  testing  activities  accomplished  on  this  day.  Also,  in  preparation  for  data  transfer  to  JADS, 
the  JADS  Network  and  Engineering  (N&E)  representatives  arrived  at  AFEWES  and  began 
installing  the  T-l  wide  area  network  (WAN)  communications  equipment. 

3.3.6  Day  #6,  4  Aug  98 

OVERVIEW:  AFEWES  operated  the  SADS  III  and  SADS  VI  while  the  JETS  replicated  the  RF 
signals  of  the  WEST  X  and  SADS  VIII.  On  this  day  of  testing,  the  SGI  Challenge  computer  had 
one  of  six  processors  fail.  The  computer  system  used  these  processors  to  run  the  AFEWES 
software  (RTI  and  data  logger  -  one  processor),  the  TAMS  model  (three  processors),  the 
interfacing  software  (interface  to  computer  simulations-one  processor),  and  Unix  operating 
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system  (one  processor).  AFEWES  tried  combining  two  functions  on  a  single  processor,  but  the 
processor  did  not  maintain  the  real-time  simulation.  Their  internal  computer  support  agency  was 
notified,  and  a  representative  for  SGI  assessed  the  situation  but  could  not  ascertain  the  problem. 
The  following  morning  a  cold  reboot  was  enough  to  clear  the  computer  fault  prompt.  We  lost 
about  two  hours  of  the  testing  day. 

RESULTS:  This  day’s  testing  yielded  a  total  of  34  runs  and  21  were  considered  usable  for 
analysis.  The  N&E  representatives  completed  the  final  T-l  installation  and  connectivity  was 
established  between  JADS  and  AFEWES. 

3.3.7  Day  #7,  5  Aug  98 

OVERVIEW:  Because  of  problems  with  the  SGI  Challenge  (problems  noted  on  day  6),  HITL 
testing  was  delayed  for  three  hours.  Finally,  the  cold  reboot  corrected  the  mysterious  processor 
fault.  The  technical  representative  from  SGI  did  not  locate  any  further  problems  and  the  system 
functioned  without  additional  faults  for  the  remaining  test  period.  After  confirming  that  there 
were  no  other  faults  indicated  by  the  SGI  Challenge,  testing  resumed  with  the  SADS  III  and 
SADS  VI  simulators.  During  initial  diagnostic  checks  of  the  SGI  Challenge,  the  SADS  III 
weapons  control  system  failed.  The  system  was  fixed,  and  no  additional  testing  time  was  lost. 
The  SADS  III  and  SGI  Challenge  systems  were  inoperative  over  the  same  period  of  time. 

RESULTS:  Although  testing  was  delayed  for  three  hours,  no  other  problems  occurred. 
AFEWES  executed  a  total  of  58  runs  and  42  were  usable  for  analysis. 

3.3.8  Day  #8,  6  Aug  98 

OVERVIEW:  JADS  continued  receiving  data  from  runs  executed  with  the  SADS  III  and  SADS 
VI  simulators  while  the  JETS  replicated  the  RF  signals  of  the  WEST  X  and  SADS  VIII.  The  only 
delay  encountered  was  caused  by  an  inoperative  optical  disk  in  one  of  the  TMCs.  Because  there 
were  several  AFEWES  gateway  shortfalls  noted  earlier,  it  became  critical  that  our  secondary  data 
collection  medium  was  operational.  This  problem  only  delayed  testing  for  an  hour.  There  was 
also  an  administrative  delay  for  an  hour. 

RESULTS:  JADS  executed  a  total  of  45  runs  and  36  were  good.  At  the  end  of  the  testing  day 
AFEWES  transmitted  the  data  collected  during  the  previous  week’s  runs  (SADS  VIII  and 
WEST-  X  manned  threat)  to  JADS. 

3.3.9  Day  #9,  7  Aug  98 

OVERVIEW:  The  last  day  of  testing  concluded  with  the  same  threat  system  configuration  as  the 
previous  day.  However,  testing  was  delayed  until  the  SADS  III  initialization  and  calibration 
processes  were  successfully  completed.  The  TAMS  also  failed  to  transmit  aircraft  position  data 
to  the  SADS  systems  on  several  occasions.  After  restarting  the  aircraft  data  scripts  and  clearing 
disk  space  on  the  SGI  Challenge,  we  still  experienced  intermittent  data  stoppage;  however,  before 
the  test  day  ended  all  the  necessary  runs  were  completed. 
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RESULTS:  We  executed  a  total  of  40  runs  and  32  were  usable  for  analysis.  All  the  remaining 
logger  data  files  were  sent  to  JADS  via  the  T-l  WAN.  Despite  more  problem  instances,  the 
second  week  was  as  successful  as  the  first  even  though  we  used  the  backup  day  to  replace  time 
lost  earlier  in  the  week. 

3.3.10  HITL  Test  Summary 

Table  15.  EW  HITL  Test  Measures  of  Performance 


MOP 

Conditions 

SADS  III 

SADS  VI 

SADS  VIII 

WEST  X 

Correct  threat  ID 

NW,SW 

X 

X 

X 

X 

Correct  threat  ID  response  time* ** 

NW,SW 

Correct  ECM  technique  selection 

NW,SW 

X 

X 

X 

X 

Correct  ECM  technique  selection 
response  time 

NW,SW 

Jamming-to-signal  ratio  versus  threats 

NW,SW 

Increase  in  tracking  error  versus  threat 

ND+NW 

SD+SW 

X 

** 

X 

X 

Number  of  breaklocks 

X 

** 

X 

X 

Reduction  in  engagement  time 

ND+NW 

SD+SW 

X 

** 

X 

X 

Reduction  in  missiles  launched 

ND+NW 

SD+SW 

X 

X 

X 

Missile  miss  distance 

ND,SD 

NW,SW 

X 

X 

X 

ND  -  northbound  dry  NW  =  northbound  wet  SD  =  southbound  dry  SW  =  southbound  wet 

*These  response  time  MOPs  could  not  be  computed,  because  the  test  was  unsuccessful  in 
accurately  recording  time-stamped  data. 

**  During  the  SADS  VI  engagements,  we  assumed  perfect  tracking. 


3.3.10.1  HITL  Run  Summary 

The  run  summary  chart  outlines  the  daily  success  rate  and  the  overall  successful  runs  completed 
during  the  HITL  test.  However,  the  in-depth  analysis  and  MOP  calculations  will  ultimately 
determine  the  final  “success”  rate  of  the  data  collected.  Estimated  completion  for  data  analysis  is 
April  1999. 
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Total  Runs 

Completed 

Aborted 

Success  Rate 

Week  1  Day  1 

31 

25 

6 

80% 

Week  1  Day  2 

51 

44 

7 

86% 

Week  1  Day  3 

40 

27 

13 

68% 

Week  1  Day  4 

42 

40 

2 

95% 

Week  2  Day  1 

0 

0 

0 

- 

Week  2  Day  2 

34 

21 

13 

62% 

Week  2  Day  3 

58 

42 

16 

74% 

Week  2  Day  4 

45 

36 

9 

80% 

Week  2  Day  5 

40 

32 

8 

80% 

Total 

341 

267 

74 

78% 

3.3.10.2  Correct  Threat  ID 

The  correct  threat  ID  MOP  measures  the  jammer’s  ability  to  correctly  identify  a  threat  system 
during  a  wet  run  as  the  threat  begins  radiating  the  target  (see  paragraph  3.2.1  for  complete 
definition  of  this  MOP).  The  initial  sample  size  for  this  MOP  equals  the  number  of  completed 
runs. 

3.3.10.3  Correct  Threat  ID  Response  Time 

This  response  time  MOP  could  not  be  computed  in  the  HITL  test  because  of  the  inability  to 
implement  an  accurate  time  signal  to  the  logger  software  within  the  AFEWES  gateway.  The 
digibus  monitor  events  could  not  be  accurately  time  synchronized  with  the  logger  file  events  to 
recreate  the  run  engagements  for  calculating  the  response  time  MOP. 

3.3.10.4  Correct  ECM  Technique  Selection 

A  correct  ECM  technique  selection  is  defined  as  the  selection  of  the  predicted  ECM  technique 
against  a  threat  on  a  wet  run.  To  assess  correct  ECM  technique  selection  accuracy,  the  only 
condition  that  must  be  met  is  a  correct  threat  ID.  Since  the  correct  ECM  selection  depends  on 
getting  a  correct  threat  ID  first,  this  sample  size  is  the  same  or  a  smaller  subset  than  the  correct 
ID. 

3.3.10.5  Correct  ECM  Technique  Selection  Response  Time 

The  digibus  monitor  events  could  not  be  accurately  time  synchronized  with  the  logger  file  events 
to  recreate  the  run  engagements  for  calculating  this  MOP  because  of  lack  of  time  synchronization 
of  the  AFEWES  computers  to  GPS  time. 

3.3.10.6  Jamming-to-Signal  Ratio  Versus  Threats 

Since  the  focus  of  this  MOP  is  on  the  potential  range  and  frequency  of  the  data  and  not 
necessarily  the  number  of  runs,  sufficient  data  were  collected  to  support  the  calculation  of  this 
MOP. 
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3.3.10.7  Increase  in  Tracking  Error  Versus  Threat 

This  MOP  involves  a  comparison  of  tracking  error  against  a  target  between  wet  and  dry  runs  with 
the  goal  of  quantifying  the  percentage  difference  between  wet  runs  and  an  established  dry  run 
baseline.  There  were  sufficient  wet  and  dry  run  data  collected  to  support  the  calculation  of  this 
MOP. 

3. 3.10.8  Number  of  Breaklocks 

The  goal  of  this  MOP  is  quantification  of  the  number  of  breaklocks  per  run  for  both  wet  and  dry 
engagements.  The  initial  data  set  was  the  number  of  runs  completed. 

3.  3.10.9  Reduction  in  Engagement  Time 

This  MOP  involves  a  comparison  of  good  engagement  time  between  wet  and  dry  runs  with  the 
goal  of  quantifying  the  percentage  difference  between  wet  runs  and  an  established  dry  run 
baseline.  Good  engagement  time  is  defined  as  the  percentage  of  time  a  threat  is  in  a  good  track 
mode  compared  to  the  total  time  from  when  the  threat  first  enters  a  good  track  mode  until  the  end 
of  the  engagement.  Tracking  error  and  track  modes  were  collected  and  should  provide  data  to 
calculate  this  MOP. 

3  3.10.10  Reduction  in  Missiles  Launched 

This  MOP  involves  a  comparison  of  the  number  of  missiles  launched  between  wet  and  dry  runs 
with  the  goal  of  quantifying  the  percentage  difference  between  wet  runs  and  an  established  dry 
run  baseline.  The  missile  shots  were  delivered  from  the  HITL  data.  Normally,  there  were  several 
missile  shots  per  completed  run. 

3.3.10.11  Missile  Miss  Distance 

The  missile  miss  distance  MOP  involves  a  comparison  of  miss  distances  between  wet  and  dry  runs 
with  the  goal  of  quantifying  the  percentage  difference  between  wet  runs  and  an  established  dry 
run  baseline.  The  missile  shots  were  collected  at  the  HITL  test. 

3.4  SIL  Execution 

The  goal  of  the  SIL  test  was  to  collect  60  response  time  samples  for  each  emitter  mode  change 
event  in  the  JADS  site  controller  matrix.  The  required  measurement  accuracy  for  the  response 
time  was  100  ms  with  a  goal  of  10  ms.  The  test  setup  is  described  in  Figure  5. 
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Figure  5.  Response  Time  Measurement  Test  Setup 


The  Automatic  Multiple  Environment  Simulator  (AMES)  was  used  to  generate  a  dynamic  RE 
scenario  that  was  used  to  stimulate  the  ALQ-131.  The  AMES  RF  output  was  coupled  into  the 
pod  behind  the  receive  antennas.  A  spectrum  analyzer  (SA)  was  used  to  monitor  the  RF  input 
into  the  pod.  A  second  spectrum  analyzer  was  used  to  monitor  the  technique  waveforms 
generated  as  responses  to  the  RF  input.  The  Warner  Robins  Air  Logistics  Center  (WR-ALC) 
version  of  the  digibus  monitor  was  connected  to  the  ALQ-131  digibus  external  connector.  A 
single  emitter  designated  as  the  synchronous  emitter  was  routed  to  the  digital  output  of  the 
AMES.  This  output  generated  a  time-to-live  (TTL)-compatible  pulse  synchronous  with  the  RF 
pulse  output  for  the  desired  emitter.  This  digital  output  was  used  to  reset  the  digibus  clock  which 
was  used  to  time-tag  R/P  events.  This  internal  digibus  clock  is  a  crystal-controlled  counter  with  a 
maximum  count  of  approximately  1 1  hours.  An  oscilloscope  was  used  to  monitor  the  digital 
output  from  the  AMES.  A  liquid  emitter  diode  (LED)  located  on  the  AMES  front  panel  was  also 
used  to  monitor  this  output.  When  an  emitter  mode  change  occurred,  the  synchronizing  emitter 
was  activated  at  the  same  time  for  a  short  interval  to  reset  the  digibus  counter.  When  the  R/P 
event  was  captured  by  the  digibus  monitor,  the  time-tag  would  then  be  the  elapsed  time  since  the 
reset  (or  the  emitter  mode  change  event  time)  which  was  the  desired  response  time. 

3.4.1  Scenario 

The  scenario  simulated  was  as  close  as  possible  to  the  one  used  during  the  OAR  baseline  testing. 
The  aircraft  heading  was  automatically  computed  by  the  AMES.  The  four  emitters  were 
simulated  using  archived  catalogs  for  threat  descriptions  and  the  actual  range  threat  positions. 
Some  changes  to  the  archive  files  were  necessary  to  simulate  the  threat.  These  changes  included 
changing  the  frequency  and  pulse  repetition  interval  (PRI)  of  the  SADS  VIII  target  track  radar 
and  removing  the  scan  from  the  SADS  VI  illuminator  signal.  These  changes  were  necessary  to 
conform  to  the  JADS  rules  of  engagement.  A  fifth  emitter  was  programmed  into  the  AMES  for 
the  purpose  of  providing  the  necessary  synchronization  to  the  digibus  monitor.  This  emitter  was 
placed  at  an  RF  frequency  outside  the  bands  of  interest  for  the  ALQ-131. 
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3.4.2  Results 


All  goals  of  the  test  were  met.  A  total  of  60  trials  were  executed  to  collect  response  time  data. 
Each  trial  consisted  of  a  southbound  and  northbound  pass  and  each  trial  generated  one  sample  per 
emitter  mode  change  event  for  a  resulting  sample  count  of  60.  Response  time  data  were  collected 
with  an  accuracy  better  than  10  ms  and  measurement  resolution  was  equal  to  or  better  than  the 
accuracy. 

3.5  Repeatability  and  Correlation  Analysis 

The  JADS  test  scenario  was  designed  to  reduce  the  run-to-run  variation.  We  scripted  as  much  of 
the  scenario  as  possible  to  limit  the  variation  within  each  facility  and  across  the  facilities.  This 
was  largely  successful.  However,  the  test  scenario  stressed  each  facility’s  ability  to  reduce  the 
run-to-run  variation.  The  primary  source  of  variation,  both  within  and  across  facilities,  was 
human  operator  and  test  control.  We  found  the  OAR  baseline  had  more  run-to-run  variation 
than  AFEWES.  A  number  of  factors  can  be  cited  for  this.  The  time  between  missions  was  a 
critical  factor.  Better  results  were  obtained  when  missions  were  flown  on  successive  days. 
Another  factor  was  the  number  of  different  operators  from  mission  to  mission  at  the  OAR. 
AFEWES  used  only  one  team  of  operators  for  each  threat.  Finally,  we  found  a  cultural  difference 
between  the  facilities.  AFEWES  is  primarily  a  developmental  test  facility,  while  the  OAR 
specializes  in  operational  effectiveness  testing.  Developmental  tests  emphasize  tight  control  and 
repeatability,  while  the  operational  tests  emphasize  operator  ffeeplay  and  skill. 

Data  results  are  classified  and  are  contained  in  a  separate  report.  GTRI  was  tasked  to  reduce  and 
analyze  all  Phase  1  data  and  report  the  results.  Their  report  will  discuss  each  MOP  and  provide 
the  statistical  results  (mean  and  variance).  JADS  will  attempt  to  identify  sources  of  variance 
where  possible.  In  addition,  GTRI  will  perform  the  correlation  analysis  of  OAR  baseline  and 
HITL  data.  The  classified  report  will  describe  the  baseline  data  that  JADS  will  use  for  correlation 
with  the  ADS-based  test  phases. 
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4.0  Lessons  Learned 


4.1  Risk  Reduction/OAR  Baseline 

4.1.1  Technical 

4.1. 1.1  Instrumentation 

This  section  summarizes  the  lessons  JADS  learned  during  the  Phase  1  execution. 

LESSONS  LEARNED:  The  overall  lesson  learned  concerning  both  range  and  airborne 
instrumentation  is  to  take  the  time  prior  to  the  test  to  thoroughly  verify  the  real  capabilities  of  the 
systems  proposed.  The  test  planner  must  be  able  to  clearly  define  specific  requirements  to  the  test 
agency  providing  the  instrumentation  and  ask  the  pertinent  questions  to  find  out  whether  the 
instrumentation  meets  the  requirement.  In  most  cases,  modifications,  improvements,  and/or 
procedural  changes  are  needed  to  tailor  the  use  of  the  instrumentation  to  the  needs  of  the 
individual  test.  Test  facilities  and  instrumentation  suppliers  are  in  the  business  of  “selling”  the 
capabilities  of  their  assets  and  it  is  up  to  the  test  manager  to  determine  their  true 
capability/applicability  by  asking  the  hard  questions.  We  were  constrained  to  existing 
instrumentation  and  as  a  result  had  to  overcome  data  shortfalls.  JADS  experience  with  the 
RDAPAS  assets  and  the  ALQ-131  1553  databus  cards  are  vivid  examples  of  this  lesson. 

4. 1.1.2  Range  Operations 
4. 1 . 1 .2. 1  Mission  Control 

Test  control  was  a  source  of  execution  problems.  Premission  problems  were  reduced  by  using  a 
very  detailed  premission  checklist  and  by  the  EW  Test  personnel  deploying  to  the  range  prior  to 
each  mission  to  confirm  preparations  were  being  implemented  according  to  plan.  During  the 
mission,  the  mission  controller,  in  charge  of  calling  out  execution  condition  “marks,  was 
sometimes  hampered  by  other  duties.  Besides  controlling  the  start  and  stop  of  each  mission,  the 
controller  was  also  expected  to  handle  real-time  equipment  problems  as  they  occurred  during  a 
mission.  Because  of  this  task  loading,  execution  “marks”  were  called  late  many  times  and 
occasionally  missed.  This  situation  improved  late  in  the  OAR  baseline  flight  schedule  when  the 
other  duties  were  finally  delegated  to  someone  else. 

There  were  also  occasional  range  coordination  problems  such  as  the  incorrect  range  area  being 
requested  for  flight  clearance,  confusion  resulting  in  video  recorders  not  to  be  turned  on, 
execution  scripts  not  available/not  reviewed  at  some  threat  sites,  and  the  test  aircraft  arriving  at 
the  IP  too  early/late  for  the  start  of  the  range  time. 

LESSON  LEARNED:  Test  controller  workload  should  be  analyzed  ahead  of  time  and  divided 
among  two  or  more  individuals  so  that  crucial  test  condition  calls  can  be  made  consistently  from 
run  to  run. 
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4. 1 . 1 .2.2  Threat  Operating  Procedures 

The  initial  JADS  approach  allowed  the  threat  operators  to  use  their  best  judgment  in  employing 
the  threats  to  acquire  and  track  the  test  aircraft.  However,  this  approach  introduced  too  much 
variation  in  the  test  conditions.  Therefore,  a  site  controller  matrix  (See  Appendix  D)  was 
developed  to  reduce  threat  operation  variability  and  generate  more  consistent  run  conditions. 
However,  it  was  realized  that  even  with  the  carefully  written  down  site  controller  matrix  and  the 
ROE,  there  was  still  some  ambiguity.  The  cure  for  these  ambiguities  was  face-to-face  meetings 
with  all  operators  early  on.  These  meetings  were  followed  by  actual  over-the-shoulder 
observation  and  pre-  and  post  briefs  by  designated  site  observers. 

An  associated  problem  involved  the  threat  operators  consistently  following  the  rules  of 
engagement.  JADS  missions  were  sometimes  3-4  weeks  apart  and  the  operators  were  not  always 
the  same  from  mission  to  mission.  This  required  JADS  to  refresh  the  operators  on  JADS-peculiar 
operations  prior  to  each  mission  to  preclude  errors  in  following  the  ROE.  This  required  a  lot  of 
extra  time  by  the  JADS  observers  in  “retraining”  the  operators. 

LESSONS  LEARNED:  A  detailed  threat  operations  matrix  will  help  ensure  consistent  threat 
tum-on/off  times,  mode  changes,  and  rules  of  engagement  procedures  for  a  structured  test  such  as 
the  JADS  EW  Test.  Also,  if  missions  are  separated  by  weeks,  the  operators  need  to  be  refreshed 
on  operating  procedures  prior  to  each  mission  to  ensure  enforcement  of  the  ROE.  This  may  not 
be  desirable  in  a  more  operational  test  and  evaluation  (OT&E)-oriented  test  but  is  necessary  when 
precise  baseline  data  are  needed.  The  communication  process  with  threat  operators  is  more 
complicated  than  it  would  appear.  For  example,  the  “no-ECCM”  ROE  seemed  clear  and 
unambiguous.  However,  only  after  a  series  of  meetings  were  all  of  the  ambiguities  cleared  up. 

4. 1.1. 2. 3  Communications 

Communications  among  JADS  personnel  in  the  mission  control  room  and  observers  at  individual 
sites  were  inadequate  at  first  but  improved  markedly  with  the  establishment  of  a  dedicated 
observer  net  in  addition  to  the  customer  net.  This  net  allowed  direct  feedback  and  test  condition 
adjustments  real  time  when  anomalies  were  noted. 

LESSON  LEARNED:  A  separate  test  observer  communications  link  between  the  mission  control 
center  and  individual  threat  sites  is  very  beneficial  to  detect  problems  and  address  them  in  near 
real  time. 

4.  L  1.3  Threat  Simulators 

There  are  no  recommendations  or  changes  associated  with  the  threat  simulators.  One  important 
concern  during  the  OAR  baseline  test  phase  was  to  ensure  that  threat  operators  consistently 
adhered  to  the  JADS  rules  of  engagement.  Because  of  the  gaps  in  time  between  risk  reduction 
missions,  large  variances  in  operator  actions  were  observed.  Operator  performance  during  a 
mission  should  be  monitored  and  variances  from  the  rules  of  engagement  recorded.  These 
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variances  should  then  be  briefed  to  the  individual  site  operators  prior  to  the  next  flight  test  mission 
to  prevent  them  from  recurring  in  subsequent  test  missions. 

4.1.1.4  Data  Processing  and  Delivery 

4. 1 . 1 .4. 1  Data  Processing  Tools 

Data  processing  was  supposed  to  be  enhanced  by  the  existence  of  an  automated  data  reduction 
application.  The  application  chosen  was  ADRS  (Automated  Data  Reduction  Software),  a  GTRI 
product  which  evolves  with  each  new  GTRI  customer  and  their  needs.  After  a  series  of 
discussions  with  the  WTR  analysts,  significant  progress  was  made  in  finalizing  data  sources, 
formats,  and  collection  rates.  However,  the  software  was  not  completely  ready  before  the 
beginning  of  testing  and  refinements  continued  beyond  Phase  1  execution. 

Our  first  ADRS  version  was  plagued  by  software  bugs  which  caused  the  system  to  crash 
frequently.  Through  diligent,  comprehensive  feedback  to  the  GTRI  programmer,  nearly  all  these 
bugs  were  fixed.  Even  though  GTRI  was  responsive,  these  fixes  took  place  midstream  and  they 
hampered  data  reduction  and  analysis. 

The  second  problem  involved  JADS  special  data  analysis  requirements  for  a  small  number  of  the 
MOPs.  More  ADRS  code  needed  to  be  built  to  handle  these  requirements.  This  recoding  took 
place  while  OAR  baseline  missions  were  being  flown.  This  overlap  impeded  reporting  final  results 
on  every  MOP. 

The  third  problem  resulted  from  changing  MOP  calculation  methods  after  receipt  of  the  first 
version  of  ADRS.  The  definition  and  calculation  of  four  MOPs  continued  to  evolve  throughout 
Phase  1  and  Phase  2  execution  to  allow  for  better  correlation  and  repeatability  in  the  test. 

The  data  analysis  and  reduction  tools  used  for  the  OAR  baseline  missions  were  cumbersome, 
often  slow  to  execute,  and  some  utilities  did  not  work  (e.g.,  time  skip  function,  MOP  data  export 
function).  The  preparatory  tools  used  on  OAR  baseline  mission  data  were  the  ALQ-131  digibus 
reader,  emitter  mode  verification  (EMV)  converter,  Dbase  utility,  and  the  Firefly  split  tool.  All  of 
these  tools  use  separate  application  interfaces  that  were  often  clumsy  and  difficult  to  understand 
(especially  the  digibus  reader).  This  made  it  very  difficult  for  new  team  members  and  outside 
personnel  to  understand  the  data  analysis  process  and  hinders  productivity. 

LESSON  LEARNED:  Contractor  developed  data  analysis  tools  should  be  completely  developed 
and  demonstrated  prior  to  the  start  of  test.  This  should  include  hands-on  practice  (training) 
reducing  and  analyzing  representative  data  expected  from  the  test.  Analysis  of  data  from  risk 
reduction  missions  will  help  work  out  the  bugs  in  the  data  analysis  tool  and  allow  for 
improvements  prior  to  working  on  data  from  the  missions  for  record. 

Data  reduction  tools  need  to  have  a  common  interface  that  will  call  upon  the  individual  tools. 
Moreover,  they  should  be  selectable  in  the  order  used  so  the  user  knows  what  actions  need  to  be 
performed  and  in  what  order. 
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4.1.1 .4.2  Data  Delivery 


Timely  data  delivery  is  paramount  to  test  success.  A  number  of  the  JADS  EW  Test  risk  reduction 
missions  were  flown  almost  back  to  back,  and  it  was  not  possible  to  obtain  and  analyze  the  results 
from  one  mission  before  flying  the  next.  Quick  data  receipt  and  analysis  can  be  crucial  to 
troubleshooting  and  mission  success,  especially  during  a  risk  reduction  process. 

LESSON  LEARNED:  The  timeline  for  flight  test  data  reduction  and  analysis  is  just  as  important 
as  the  mission  schedule  timeline.  Efficiency  is  lost  (i.e.,  successive  missions  flown  with  bad 
assets)  if  data  cannot  be  examined  between  missions.  Missions  (especially  during  risk  reduction) 
must  be  scheduled  with  adequate  time  between  to  analyze  data  from  the  previous  mission. 
Another  lesson  is  that  all  means  must  be  exercised  to  expedite  delivery  of  range  and  aircraft  data 
products  to  the  analysts  whenever  mission  schedules  are  tight. 

4.1.2  Infrastructure 

4. 1.2.1  Personnel 

The  JADS  EW  Test  team  is  comprised  of  a  few  individuals  with  expertise  in  areas  of  program 
management,  test  and  evaluation,  electronic  warfare,  analysis,  statistics,  etc.  Additional  analyst 
support  is  provided  within  the  JADS  Joint  Test  Force  outside  of  the  EW  Test  team.  All  key 
personnel  with  the  exception  of  GTRI  were  collocated.  Integrating  GTRI  personnel  into  the  daily 
decision-making  process  was  elusive. 

LESSON  LEARNED:  The  mix  of  skills  the  team  possessed  was  adequate  for  managing  the  task. 
JADS  was  fortunate  to  get  properly  skilled  military  and  contractor  people  for  key  positions. 
Future  test  organizations  need  to  plan  on  filling  highly  technical  or  special  skills  positions  with 
contractors  or  through  arrangements  with  outside  agencies. 

4. 1.2.2  Support  Agencies 

4. 1 .2.2. 1  Integrated  Product  Team  (IPT) 

The  IPT  helped  bridge  gaps  in  expertise  and  helped  with  programmatic  decisions.  The  concept  of 
an  IPT  is  a  good  one.  However,  disadvantages  arise  with  the  geographical  separation  of  the 
members  of  the  IPT.  The  JADS  EW  Test  IPT  consists  of  members  resident  from  coast  to  coast. 
Teleconferencing  and  electronic  mail  are  powerful  tools  that  overcame  some  of  these 
communication  obstacles;  however,  the  most  progress  is  made  when  members  meet  face-to-face. 
This  is  due  to  the  highly  technical  nature  of  the  work,  the  concepts  involved,  and  the 
software/hardware  which  are  used  on  this  test. 

LESSONS  LEARNED:  When  technical  problems  arise,  teleconferences  and/E-mail  can  help 
solve  many  of  the  issues,  but  in  many  cases  face-to-face  meetings  between  key  players  are  needed 
to  resolve  items.  This  allows  direct  give-and-take  questions  to  reduce  incorrect  interpretations 
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and  confusion  of  what  was  said  versus  what  was  meant.  Such  meetings  must  have  the  following 
elements. 

•  Attendees  limited  to  those  individuals  from  each  key  agency  who  can  and  will  make  decisions 
and  commit  required  support  to  the  test 

•  Specific  agenda  including  status  updates  from  previous  actions  items 

•  Specific  action  items  assigned  to  individuals  by  name  with  suspense  dates 

•  Periodic  review  and  formal  tracking  of  action  item  suspenses 

4.1. 2.2.2  AATC 

Normally,  operations  went  smoothly  in  this  area.  JADS  received  outstanding  support  from 
AATC  ECM  pod  personnel.  However,  occasionally  data  were  lost  during  some  of  the  test 
missions  because  of  errors  in  pod  control  switch  actions  in  flight.  This  appeared  to  be  because  of 
pilot  unfamiliarity  and/or  lack  of  premission  briefing  on  the  pod  functions  and  controls.  A  number 
of  different  pilots  flew  the  various  test  missions,  and  some  were  very  familiar  with  ALQ- 131  pod 
operation  and  peculiarities  while  others  were  not. 

At  times,  JADS  had  difficulty  effectively  scheduling  AATC  test  aircraft,  ECM  pods,  and 
personnel  to  support  available  test  range  periods.  Scarce  assets  and  other  AATC  mission 
priorities  as  well  as  late  changes  to  range  schedules  contributed  to  this  problem.  JADS 
compensated  by  over  scheduling  the  range. 

LESSON  LEARNED:  The  aircrew  must  be  thoroughly  briefed  on  operation  and  functions  of  the 
system  under  test.  This  is  especially  crucial  when  test  instrumentation  is  interfaced  with  the  test 
item  that  may  affect  its  normal  operation.  Whenever  feasible  within  aircrew  scheduling 
limitations,  the  same  aircrew  members  should  fly  as  many  of  the  test  missions  as  possible  to 
reduce  the  learning  curve  from  mission  to  mission. 

The  lesson  learned  concerning  scheduling  problems  is  that  test  managers  must  get  firm  support 
agency  commitments  during  initial  test  planning,  fully  understand  support  agency  and  range 
scheduling  flexibility  limitations,  and  factor  those  into  the  overall  test  schedule.  However,  over 
scheduling  has  negative  impact  on  efficient  range  usage. 

4.1. 2.2.3  GTRI 

GTRI  technical  expertise  was  crucial  to  the  success  of  the  JADS  EW  Test  program.  GTRI  direct 
interaction  with  JADS  EW  Test  team  members  was  needed  to  address  a  number  of  data  analysis 
issues  during  risk  reduction.  Lack  of  effective  communication  among  individuals  during  this 
process  sometimes  was  a  problem  which  impacted  solving  issues.  The  technical  expertise 
required  was  resident  within  the  GTRI  staff;  however,  it  was  difficult  to  access  it  because  early 
agreements  established  a  single  point  of  contact  (POC)  (GTRI  program  manager)  to  coordinate 
the  contracted  technical  expertise.  Because  of  this  bottleneck,  the  team  was  often  unable  to 
directly  interface  with  the  individual  GTRI  expert  needed  to  solve  a  particular  problem  in  a  timely 
manner.  This  also  unnecessarily  burdened  the  program  manager  with  personnel  management 
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actions  instead  of  allowing  time  to  work  technical  problems  effectively.  Critical  details  were  lost 
transmitting  information  through  multiple  nodes  in  the  organization  and  progress  was  impeded. 
This  led  to  many  miscommunications,  misunderstandings,  delays,  and  undue  stress  on  the  GTRI 
POC  and  JADS  team  members. 

LESSON  LEARNED:  Having  a  contractual  single  point  of  contact  to  coordinate  support  can  be 
a  good  management  technique;  however,  that  POC  must  be  flexible  enough  to  arrange  for  and 
allow  direct  working  relationships  among  technical  experts  and  the  test  team  members.  Such 
agreements  should  be  established  early  in  the  program  and  clearly  defined  to  the  satisfaction  of  all 
parties. 

4 . 1 . 2 . 2 .4  J ammer  Support 

AWC  support  was  required  throughout  the  risk  reduction  effort  in  several  areas  including 
technical  expertise,  troubleshooting,  and  instrumentation  hardware  and  software.  This  support 
was  provided  without  fail,  many  times  on  short  notice  and  requiring  the  help  of  several  internal 
AWC  organizations.  AWC  personnel  support  and  instrumentation  recorder  cards,  even  though 
not  formally  coordinated  prior  to  the  test,  were  exceptional  and  precluded  several  major  test 
delays. 

WR-ALC  technical  support  was  provided  in  the  test  planning  phase  and  was  available  throughout 
the  risk  reduction  phase.  Interface  difficulties  with  the  WR-ALC-developed  1553  bus  recorder 
cards  prevented  their  use  during  the  majority  of  this  phase  because  of  the  length  of  time  that 
would  have  been  required  to  refine  the  interface.  Data  collected  on  the  ALQ-131  recorder  cards 
precluded  the  need  for  rework  of  the  1553  bus  recorder  cards. 

LESSON  LEARNED:  Given  the  low  priority  of  a  joint  test  activity,  the  tester  needs  to 
investigate  all  possible  sources  of  support  and  technical  expertise  for  the  system  under  test.  In 
our  case  we  received  better  support  from  the  operational  rather  than  logistics  agencies. 

4.2  HITL 

4.2.1  Technical 

4. 2. 1.1  AFEWES  Operations 

JADS  and  AFEWES  had  not  firmly  established  the  type  of  JADS  user  accounts  and  level  of 
permissions  required  for  SGI  Challenge  computer.  Moreover,  a  location  for  the  HITL  data 
(computer  system,  file  labeling  convention)  at  JADS  was  not  established  prior  to  data  transfer. 
Once  the  data  were  transferred  to  JADS,  the  location  was  known  only  to  certain  individuals 
making  finding  the  data  cumbersome.  AFEWES  also  used  a  confusing  file  labeling  convention  for 
each  run — the  file  contained  the  date  and  time  but  not  the  run  number.  AFEWES  changed  the 
labels  when  the  data  were  reconstructed  from  the  optical  disk  format.  In  future  testing  the  file 
labeling  convention  will  be  clearly  defined  with  no  ambiguity. 
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LESSON  LEARNED:  Testers  must  establish  user  accounts  and  file  naming  convention  prior  to 
starting  a  test. 

4.2.1. 2  Threat  Simulators 

If  differences  in  the  Western  Test  Range  SADS  VI  and  the  AFEWES  SADS  VIM  had  been 
sufficiently  understood  by  JADS  and  AFEWES,  the  tracking  error  problem  may  have  consumed 
less  time  to  resolve.  Generally,  there  were  several  differences  in  the  operation  and  design  of  the 
Western  Test  Range  and  AFEWES  simulators  that  were  not  fully  understood  until  after  the 
completion  of  the  Phase  1  OAR  baseline  test.  Although  the  ROE  were  designed  to  bridge  these 
differences,  we  could  not  constrain  the  ROE  for  the  HITL  test  any  further.  The  operational 
differences  noted  during  the  OAR  baseline  test  were  beyond  the  AFEWES  threat  simulators’ 
system  limitations. 

LESSON  LEARNED:  To  reduce  differences  between  WTR  and  AFEWES  (or  between  any  two 
test  facilities)  the  tester  should  sponsor  an  open  interchange  among  system  experts.  In  our  case 
we  needed  the  operators  of  the  threat  systems  from  each  test  facility  to  meet  and  identify  the 
correct  ROE. 

4. 2. 1.3  Data  Processing 

The  integration  of  ADRS  into  the  data  analysis  process  was  slow.  Moreover,  the  AFEWES 
nonstandard,  newly  developed  software,  RTI  interface  logger,  and  the  ADRS  tool  were  a 
hastened  design  and  integration  process.  The  time  required  to  develop  and  integrate  these  items 
was  under  estimated  and  limited  by  the  condensed  schedule.  Although  the  logger  was  helpful  for 
the  next  phase  of  testing,  JADS  later  determined  that  the  data  published  by  the  AFEWES 
software  were  still  inadequate  for  analysis — critical  items  remained  missing.  However,  the  valid 
data  were  later  retrieved  from  AFEWES  optical  disk  format  and  processed  through  their  amended 
federate  software.  Additionally,  the  HITL  test  was  designed  to  have  ADRS  available  for 
processing  the  very  first  set  of  test  data;  however,  this  did  not  happen.  ADRS  was  not  available 
to  review  the  data  for  completeness  and  to  ensure  the  logger  and  digibus  monitor  files  were  time 
synchronized;  therefore,  the  SIL  test  was  required  to  be  added  to  collect  the  jammer  timing 
MOPs. 

LESSON  LEARNED:  The  HITL  test  used  several  new  or  modified  software  components  as  a 
part  of  data  collection  and  analysis.  Under  these  conditions,  it  is  paramount  that  testers  exercise 
each  component  and  verify  that  it  conforms  to  the  agreed  upon  specifications  even  within  a 
condensed  schedule.  Furthermore,  data  analysis  tools  should  be  verified  prior  to  the  execution  of 
the  test. 

4.2.2  Infrastructure 

The  necessity  for  accurate  timing,  given  our  testing  and  architectural  design,  was  paramount; 
however,  there  was  a  gap  between  what  JADS  conveyed  and  what  AFEWES  comprehended. 
JADS  was  the  first  customer  at  AFEWES  to  require  that  all  data  sources  have  a  common  time 
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reference  outside  of  their  normal  resources.  Although  site  surveys  and  technical  interchange 
meetings  occurred,  there  remained  a  gap  in  understanding  of  the  timing  requirements. 

LESSON  LEARNED:  The  customer  must  ensure  that  all  requirements  are  met.  A  requirements 
back-brief  or  critical  design  review  from  AFEWES  might  have  prevented  this  misunderstanding. 
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Appendix  A  -  Acronyms,  Abbreviations  and  Definitions 


413  FLTS 

A/C 

AAA 

AATC 

ACC 

ACETEF 

ACVR 

ADRS 

ADS 

AFB 

AFEWES 

AGC 

AIM 

ALQ-131 


ALR-69 

AM 

AMES 

APA 

AWC 

A Z 

C4ISR 

CW 

dB 

DDT&E 

DMAP 

DO 

DoD 

dry  run 

DSM 

DT&E 

DTM 

DTMS 

ECCM 

ECM 

EIOB 

E-mail 


413th  Flight  Test  Squadron,  Edwards  AFB,  California 
aircraft 

anti-aircraft  artillery 

Air  National  Guard  Air  Force  Reserve  Test  Center,  Tucson,  Arizona 
Air  Combat  Command 

Air  Combat  Environment  Test  and  Evaluation  Facility,  Patuxent  River, 

Maryland;  Navy  facility 

advanced  crystal  video  receiver 

Automated  Data  Reduction  Software 

advanced  distributed  simulation 

Air  Force  base 

Air  Force  Electronic  Warfare  Evaluation  Simulator,  Fort  Worth,  Texas; 
Air  Force  managed  with  Lockheed  Martin  Corporation 
automatic  gain  control 
air  intercept  missile 

a  mature  self-protection  jammer  system;  an  electronic  countermeasures 
system  with  reprogrammable  processor  developed  by  Georgia  Tech 
Research  Institute 

airborne  multipurpose  radar  warning  receiver  used  in  F-16  aircraft 
amplitude  modulation 

Automatic  Multiple  Environment  Simulator  at  Eglin  AFB,  Florida 
analysis  plan  for  assessment 

Air  Warfare  Center  at  Nellis  AFB,  Nevada,  and  Eglin  AFB,  Florida 
azimuth 

command,  control,  communications,  computers,  intelligence,  surveillance 
and  reconnaissance 
continuous  wave 
decibel 

Deputy  Director,  Test  and  Evaluation 
data  management  and  analysis  plan 
deputy  commander  for  operations 
Department  of  Defense 
the  system  under  test  is  off 
digital  system  model 
developmental  test  and  evaluation 
digibus  traffic  monitor 
Digibus  Traffic  Monitor  System 
electronic  counter-countermeasures 
electronic  countermeasures 
enhanced  input/output  buffer 
electronic  mail 
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EL 

elevation 

EMV 

emitter  mode  verification 

EW/EW  Test 

electronic  warfare;  JADS  Electronic  Warfare  Test 

FCS 

fire  control  system 

ft  or’ 

feet 

GPS 

global  positioning  system 

GTRI 

Georgia  Tech  Research  Institute,  Atlanta,  Georgia 

GUI 

graphical  user  interface 

HITL 

hardware-in-the-loop  (electronic  warfare  references) 

HLA 

high  level  architecture 

hr 

hour 

Hz 

hertz 

I/O 

input/output 

IADS 

Integrated  Air  Defense  System 

IAW 

in  accordance  with 

IC 

initial  condition 

ICD 

interface  control  document 

ID 

identification 

IF 

intermediate  frequency 

IFF 

identification  friend  or  foe 

INS 

inertial  navigation  system 

IP 

internet  protocol;  initial  point 

IPT 

integrated  product  team 

IRIG 

Inter-Range  Instrumentation  Group 

ISTF 

installed  system  test  facility 

J/S 

jamming-to-signal  ratio 

J+S 

jamming  plus  signal 

JADS 

Joint  Advanced  Distributed  Simulation,  Albuquerque,  New  Mexico 

JETS 

JammEr  Techniques  Simulator 

JT&E 

joint  test  and  evaluation 

JTF 

Joint  Test  Force,  Albuquerque,  New  Mexico 

K 

thousand 

kHz 

kilohertz 

LED 

liquid  emitter  diode 

m 

meter 

MCC 

mission  control  center 

MHz 

megahertz 

MIL-STD 

military  standard 

mm 

millimeter 

MMDS 

missile  miss  distance  simulation 

MOP 

measure  of  performance 

ms 

millisecond 

MSL 

mean  sea  level 

MSN 

mission 

MT 

message  tape 
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N&E 

network  and  engineering 

NM 

nautical  mile 

ns 

nanosecond 

OAR 

open  air  range 

OFP 

operational  flight  program 

OSD 

Office  of  the  Secretary  of  Defense 

OT&E 

operational  test  and  evaluation 

PC 

personal  computer 

POC 

point  of  contact 

PRF 

pulse  repetition  frequency 

PRI 

pulse  repetition  interval 

PT 

preflight  message  tape 

pip 

program  test  plan 

QA 

quality  assurance 

R/P 

receiver  processor 

RAJPO 

Range  Applications  Joint  Program  Office,  Eglin  Air  Force  Base,  Florida 
(provides  GPS  pods) 

RCS 

radar  cross-section 

RDAPAS 

Radar  Detection  and  Performance  Analysis  System 

RF 

radio  frequency 

ROE 

rules  of  engagement 

RTC 

reference  test  condition 

RTI 

runtime  infrastructure 

RWR 

radar  warning  receiver 

S/W 

software 

SA 

situational  awareness;  spectrum  analyzer 

SAC 

senior  advisory  council 

SADS 

Simulated  Air  Defense  System 

SAIC 

Science  Applications  International  Corporation 

SAM 

surface-to-air  missile 

SASY 

Signal  Analysis  System 

SGI 

Silicon  Graphics,  Inc. 

SIL 

system-in-the-loop;  system  integration  laboratory 

SIMVAL 

simulation  validation 

SPAG 

software-programmable  antenna  pattern  generator 

Spectrum™ 

an  instrumentation  suite  used  to  measure  band  width  utilization 

SPJ 

self-protection  jammer 

SSA 

SASY  site  Alpha 

SUT 

system  under  test 

SVB 

signal  van  Bravo 

SVC 

signal  van  Charlie 

SWR 

Southwest  Research  Corporation 

T&E 

test  and  evaluation 

T/O 

take  off 

T-l 

digital  carrier  used  to  transmit  a  formatted  digital  signal  at  1 .544  megabits 
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per  second 

TAB 

technical  advisory  board 

TAMS 

Tactical  Air  Mission  Simulator 

TAP 

test  activity  plan 

TAR 

target  acquisition  radar 

TATS 

Target  Acquisition  and  Tracking  System 

TCAC 

Test  Control  and  Analysis  Center,  Albuquerque,  New  Mexico 

TDOP 

TSPI  (time-space-position  information)  Data  Optimizing  Processor 

TEL 

transporter,  erector,  launcher 

TMC 

test  management  center 

TSPI 

time-space-position  information 

TTL 

time-to-live 

TTR 

target  tracking  radar 

UTC 

universal  time  code 

V&V 

verification  and  validation 

VCR 

video  cassette  recorder 

VHS 

electronic  system  for  recording  video  and  audio  information 

WAN 

wide  area  network 

WEST 

Weapon  Evaluation  Simulated  Threat 

wet  run 

the  system  under  test  is  on 

WR-ALC 

Warner  Robins  Air  Logistics  Center,  Georgia 

WTR 

Western  Test  Range 
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Appendix  B  -  JADS  EW  Test  Rules  of  Engagement 


Introduction 

This  appendix  was  prepared  to  delineate  the  common  rules  of  engagement  (ROE)  that  were  used 
during  the  open  air  range  (OAR)  baseline  testing  and,  where  appropriate,  during  remaining  phases 
of  the  Joint  Advanced  Distributed  Simulation  (JADS)  Electronic  Warfare  (EW)  Test.  ROE  are 
intended  to  constrain  the  operator  actions  to  those  that  can  be  accomplished  at  both  test  locations 
(open  air  range  and  the  Air  Force  Electronic  Warfare  Evaluation  Simulator)  while  allowing  some 
freedom  to  engage  the  aircraft.  The  intent  is  to  activate  and  deactivate  the  threat  sites  at  the  same 
range  for  every  pass.  A  script  was  developed  to  queue  each  site  operator  when  to  radiate  and 
when  to  go  to  standby  modes.  The  site  is  expected  to  acquire,  track,  and  simulate  firing  at  the 
target  within  the  rules  provided  below.  Prior  to  activation  and  after  deactivation  the  site  is 
expected  to  be  in  “standby/dummy  load.”  These  rules  are  necessary  to  allow  maximum 
opportunity  for  gathering  repeatable  data  across  all  test  phases  while  allowing  enough  variation 
between  engagements  to  assess  the  impact  of  advanced  distributed  simulation  technology. 

Bl.  Simulated  Air  Defense  System  (SADS)  III  ROE 

Bl.l  General 

•  The  operator  will  ensure  the  system  does  not  use  any  electronic  counter-countermeasures 
(ECCM)  at  any  time.  Ensure  instrumentation  is  reporting  miss  distance  and  missile  position 
data. 

•  High  pulse  repetition  frequency  (PRF)  will  be  used  for  target  acquisition  and  tracking. 

•  One  engagement  will  be  flown  per  each  northbound  and  southbound  leg  of  the  profile. 

•  Threat  engagements  will  be  scripted  (site  operator  “radiate/standby”  times  directed  by 
ramrod). 

•  Target  Acquisition  and  Tracking  System  (TATS)  will  be  used  prior  to  each  threat  engagement 
to  ensure  the  target  is  within  both  the  azimuth  (AZ)  and  elevation  (EL)  beams  of  the  target 
tracking  radar  (TTR)  system.  TATS  will  be  turned  off  just  prior  to  the  operator  initiating 
radar  track. 

•  The  operator  will  engage  the  target  as  often  as  normal  but  not  exceed  six  salvos  per 
engagement  window  with  several  seconds  between  last  missile  flyout  (recall)  and  launch  of  the 
next  salvo.  This  is  because  of  a  limitation  with  another  simulator  during  later  phases  of  the 
JADS  EW  Test. 

•  Site  operators  may  use  TATS  to  reacquire  a  lost  target  track  if  other  attempts  to  reacquire  fail 
after  a  reasonable  amount  of  time  (~several  seconds). 

•  3 -point  guidance  mode  (beam  riding)  is  the  only  guidance  mode  allowed. 

•  Radar  Detection  and  Performance  Analysis  System  (RDAPAS)  instrumentation  will  be  on  at 
all  times  the  system  is  radiating. 

•  When  your  “OFF”  mark  is  called,  immediately  go  to  standby/dummy  load  even  if  missiles  are 
in  the  air. 
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B1.2  Normal  Threat  Engagement  Sequence 

•  Site  in  standby/dummy  load  and  slaved  to  target  through  TATS. 

•  At  the  appropriate  “MARK”  call,  site  operator  will  disengage  TATS,  go  out  of 
standby/dummy  load,  and  acquire  the  target. 

•  The  threat  will  attempt  to  acquire  target  using  high  PRF  mode  via  the  trough  antenna. 

•  The  site  operator  auto  tracks  the  target  (manually  aided  if  necessary)  using  high  PRF  mode 
when  radiating  through  the  antenna  dish  and  receiving  through  the  antenna  trough. 

•  Operator  fires  missiles  at  will  not  to  exceed  six  salvos  per  engagement. 

•  After  all  missile  engagements  have  been  completed,  the  site  continues  to  track  the  target  until 
commanded  to  disengage.  At  this  time,  the  site  returns  to  standby/dummy  load  and  prepares 
for  next  engagement. 

B2.  SADS  VI  ROE 

B2.1  General 

•  The  operator  will  ensure  the  system  does  not  use  ECCM  at  any  time. 

•  Miss  distance/missile  position  data  from  the  OAR  baseline  test  phase  is  not  required  (JADS 
will  obtain  missile  launch  times  from  the  OAR  and  use  standard  seeker  data  products  from 
AFEWES  for  all  the  test  phases). 

•  One  threat  engagement  will  be  flown  per  each  northbound  and  southbound  leg  of  the  profile. 

•  Threat  engagements  will  be  scripted  (site  operator  “radiate/standby”  times  directed  by 
ramrod). 

•  TATS  will  be  used  prior  to  each  threat  engagement  to  ensure  the  target  is  within  the  TTR 
beam  of  the  system. 

•  The  operator  will  engage  the  target  as  often  as  normal.  Site  operators  may  use  TATS  to 
reacquire  a  lost  target  track  if  attempts  to  reacquire  fail  after  a  reasonable  amount  of  time 
(-several  seconds). 

•  Missile  launches  will  be  directed  by  the  command  vehicle  not  the  launch  vehicle. 

•  A  maximum  of  6  launches  per  engagement  with  at  least  30  seconds  from  last  missile  flyout 
(recall)  to  launch  of  next  missile  will  be  used.  No  more  than  one  simulated  missile  in  the  air  at 
a  time. 

•  The  illuminator  signal  should  be  on  during  each  missile  launch  and  off  at  the  end  of  the  missile 
flyout. 

•  When  your  “OFF”  mark  is  called,  immediately  go  to  standby/dummy  load  even  if  missiles  are 
in  the  air. 
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B2.2  Normal  Threat  Engagement  Sequence 

•  The  site  will  be  in  standby/dummy  load  and  slaved  to  the  target  through  TATS. 

•  At  the  appropriate  “MARK”  call,  site  operator  will  disengage  TATS,  go  out  of 
standby/dummy  load,  and  acquire  the  target. 

•  The  operator  will  use  the  target  acquisition  radar  (TAR)  to  acquire  the  target. 

•  TTR  radiates  and  tracks  the  target. 

•  The  site  operator  engages  the  target  as  normal  under  command  vehicle  control. 

•  After  all  missile  engagements  have  been  completed,  the  site  continues  to  track  the  target  until 
commanded  to  disengage.  At  this  time,  the  site  returns  to  standby/  dummy  load  and  prepares 
for  the  next  engagement. 

B3.  SADS  VIII  ROE 

B3.1  General 

•  The  operator  will  ensure  the  system  does  not  use  ECCM  at  any  time.  Ensure  instrumentation 
is  reporting  miss  distance  and  missile  position  data. 

•  No  staggered  pulse  repetition  interval  (PRI)  modes  will  be  allowed.  Site  operators  will  use 
normal  PRI  mode  for  target  tracking. 

•  One  threat  engagement  will  be  flown  per  each  northbound  and  southbound  leg  of  the  profile. 

•  Optical  tracking  will  be  allowed  only  to  augment  TATS. 

•  The  operator  will  operate  the  TAR  in  normal  mode  (360  degree  search). 

•  Threat  engagements  will  be  scripted  (site  operator  “radiate/standby”  times  directed  by 
ramrod). 

•  The  operator  will  engage  the  target  as  often  as  normal. 

•  A  maximum  of  8  launches  (missiles)  per  engagement  with  several  seconds  between  last  missile 
flyout  (recall)  and  next  launch  will  be  used.  No  more  than  2  simulated  missiles  in  the  air  at  a 
time. 

•  Site  operators  may  use  TATS  (with  optics)  to  reacquire  a  lost  target  track  if  attempts  to 
reacquire  fail  after  a  reasonable  amount  of  time  (~several  seconds). 

•  3 -point  guidance  mode  (beam  rider)  is  the  only  guidance  mode  allowed. 

•  RDAPAS  instrumentation  will  be  on  at  all  times  the  system  is  radiating. 

•  When  your  “OFF”  mark  is  called,  immediately  go  to  standby/dummy  load  even  if  missiles  are 
in  the  air. 
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B3.2  Normal  Threat  Engagement  Sequence 

•  The  site  will  be  in  standby/dummy  load  and  slaved  to  the  target  through  TATS  (with  optics). 

•  At  the  appropriate  “MARK”  call,  site  operator  will  disengage  TATS,  go  out  of 
standby/dummy  load,  and  acquire  the  target. 

•  TAR  will  be  used  to  initially  acquire  the  target.  The  site  operator  transitions  to  TTR  using 
normal  threat  hand-off  doctrine  ( i.e.,  operator’s  discretion). 

•  The  site  operator  engages  the  target  as  normal. 

•  After  all  missile  engagements  have  been  completed,  the  site  continues  to  track  the  target  until 
commanded  to  disengage.  At  this  time,  the  site  returns  to  standby/  dummy  load  and  prepares 
for  the  next  engagement. 

B4.  Weapon  Evaluation  Simulated  Threat  (WEST)  X  ROE 

B4.1  General 

•  The  operator  will  ensure  the  system  does  not  use  ECCM  at  any  time. 

•  No  miss  distance/projectile  position  data  will  be  required. 

•  No  stagger  PRI  modes  will  be  allowed.  The  normal  PRI  mode  should  be  used  during  all 
engagements. 

•  Threat  engagements  will  be  flown  on  northbound  runs  only. 

•  Threat  engagements  will  be  scripted  (site  operator  “radiate/standby”  times  directed  by 
ramrod). 

•  The  site  operator  will  use  radar  data  only;  optical  tracking  will  be  allowed  only  to  augment 
TATS. 

•  TATS  (assisted  by  optics)  will  be  used  prior  to  each  threat  engagement  to  ensure  the  target  is 
within  the  beam  of  the  TTR.  TATS/optics  can  be  used  until  after  auto  track  is  established, 
then  TATS  will  be  turned  off  and  optics  not  used. 

•  Site  operators  may  use  TATS  (aided  by  optics)  to  reacquire  a  lost  target  track  if  other 
attempts  to  reacquire  fail  after  a  reasonable  amount  of  time  (~several  seconds). 

•  RDAPAS  instrumentation  will  be  on  at  all  times  the  system  is  radiating. 

B4.2  Normal  Threat  Engagement  Sequence 

•  The  site  will  be  in  standby/dummy  load  and  slaved  to  the  target  through  TATS  with  the 
assistance  of  optics. 

•  The  site  operator  will  attempt  to  acquire  the  target. 

•  Immediately  after  establishing  auto  track,  TATS/optics  will  be  disengaged. 

•  The  site  operator  auto  tracks  the  target  (manually  aided  if  necessary). 

•  The  site  operator  engages  the  target  as  normal.  The  site  continues  to  track  the  target  until 
commanded  to  disengage.  At  this  time,  the  threat  returns  to  standby/dummy  load  and 
prepares  for  the  next  engagement. 
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Appendix  C  -  Go/No-Go  List. 


Resource 


Go/No-Go  Criteria 


|  Mission  Personnel 

Range  Ramrod 

Required 

Range  Aircraft  Controller 

Required 

JADS  Mission  Controller 

Required 

JADS  Mission  Observers 

Not  Required  -  Quick-look  and  full  analysis  can  still  be  done 

GTRI  SASY  Observer 

Required  at  SSA  or  SVC;  not  required  at  SVB 

Threat  Site  Operators 

Required  at  each  operational  threat  site  (see  “Threat  Sites” 
below) 

AATC  Observer 

Not  required 

AATC  =  Air  National  Guard  Air  Force  Reserve  Test  Center 

SASY  -  Signal  Analysis  System 

SVB  =  signal  van  Bravo 

GTRI  -  Georgia  Tech  Research  Institute 

SSA  =  SASY  site  Alpha 

SVC  =  signal  van  Charlie 

|  Aircraft  Systems 

F- 16  Test  Aircraft 

Required  -  A/C  #1262.  Must  have  all  proper  stores  (ALQ-13 1 
on  centerline  station,  2  RAJPO  pods  or  1  RAJPO  and  1  AIM-9 
simulator  pod,  2  fuel  tanks,  and  1  Firefly). 

F- 16  INS 

Required. 

F-16  FCS  Radar 

Not  Required 

ALQ-131  Jammer 

Required  -  Pod  #633;  All  3  bands  must  be  operational. 

ALQ-131  Mission  S/W 

Load 

Required  -  limited  receive/limited  response  tape  (MT-0701) 

ALR-69  RWR 

Required  -  ACVR  hardware  preferred 

ALR-69  Video  Recording 

Not  Required 

Firefly  Pod 

Required  -  on  station  #7 

ALQ-131  Recorder  Cards 

Required 

RAJPO  Pods 

1  Required  -  2  pods  preferred 

Pilot  Flight  Cards 

Required 

A/C  =  aircraft 

AIM  =  air  intercept  missile 

FCS  =  fire  control  system 

MT  =  message  tape 

RWR  =  radar  warning  receiver 

ACVR  =  advanced  crystal  video  receiver 

ALQ-131  =  a  mature  self-protection  jammer  system 

INS  =  inertial  navigation  system 

RAJPO  =  Range  Applications  Joint  Program  Office 

S/W  =  software 
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GO  /  NO-GO  List  (Continued) 


Resource 


Go/No-Go  Criteria 


1  Range  Support 

Threat  Sites 

Required  -  Note:  all  4  sites  preferred.  As  a 
minimum,  3  sites  must  be  operational.  j 

Switch  Activity  Records 

Required 

MMDS  Instrumentation 

Required  -  Missile  miss  distance 
instrumentation  must  be  available  for  SADS  III 
and  SADS  VIII. 

Threat  Video  Recording 

SASY  Site  SSA 

Not  Required 

Required  -  Note:  SSA  not  required  if  Spectrum, 
pulse,  and  network  analyzers  can  be  relocated  to 
SASY  SVC. 

SASY  Van  SVB 

Not  Required 

SASY  Van  SVC 

Not  Required 

SASY  Pulse  Descriptors 

Required 

SASY  Video  Recording 

Required  -  SADS  VI  is  especially  critical. 

RDAPAS  Instrumentation 

Required  -  Note:  An  individual  mission  will  not 
be  aborted  if  one  of  the  three  RDAPAS  systems 
is  inoperable  for  that  mission;  however, 
substantial  RDAPAS  data  must  be  collected  for 
SADS  III ,  SADS  VIII,  and  WEST  X  by  end  of 
OAR  baseline  phase. 

TATS 

Required  -  at  all  threat  sites  and  SASY  SSA  and 
SVC  sites. 

Radar  Tracking  TSPI 

Not  Required 

Observer  Room  Displays 

Not  Required 

Test  A/C  Ground  Refueling 

Not  Required 

MMDS  =  missile  miss  distance  simulation 

RDAPAS  =  Radar  Detection  and  Performance  Analysis  System 

Spectrum™  =  an  instrumentation  suite  used  to  measure  band  width  utilization 

TSPI  =  time-space-position  information 

OAR  =  open  air  range 

SADS  =  Simulated  Air  Defense  System 

TATS  =  Target  Acquisition  and  Tracking  System 

WEST  =  Weapon  Evaluation  Simulated  Threat 
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Appendix  D  -  Site  Controller  Matrix 


SADS  =  Simulated  Air  Defense  System 
WEST  =  Weapon  Evaluation  Simulated  Threat 


Condition 

Range 
(nautical 
miles  from 
initial  point) 

SADS  III 

SADS  VI 

SADS  VIII 

WEST  X 

Northbound 

1 

ON 

OFF 

OFF 

OFF 

2 

4.5 

66 

ON 

66 

66 

3 

6.0 

66 

66 

ON 

66 

4 

8.6 

cc 

66 

66 

ON 

5 

13.6 

u 

66 

OFF 

66 

6 

OFF 

66 

66 

66 

7 

17.6 

u 

66 

OFF 

Southbound 

1 

1.5 

OFF 

OFF 

ON 

OFF 

2 

3.7 

66 

ON 

66 

66 

3 

6.0 

ON 

66 

66 

66 

4 

16.0 

C4 

66 

OFF 

66 

5 

17.5 

66 

66 

66 

6 

21.0 

OFF 

66 

66 

66 
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Appendix  E  -  Range  Instrumentation  Description  and  Limitations 


El.  Signal  Analysis  System  (SASY) 

El.l  Description 

The  SASY  and  associated  signal  vans  provide  radio  frequency  (RF)  measurement  capabilities  for 
collecting  quality  assurance  (QA)-type  data  relative  to  the  RF  test  environment.  SASY  performs 
this  function  by  using  multiple  sites  and  multiple  pieces  of  RF  collection  equipment  located  at 
each  site.  The  primary  functions  of  the  SASY  relative  to  the  Joint  Advanced  Distributed 
Simulation  (JADS)  test  are 

•  Provide  QA  verification  of  intended  RF  emissions  from  the  threat  sites.  The  SASY  is 
designed  to  verify  that  all  sites  are  operating  in  their  intended  modes  as  defined  by  the  rules  of 
engagement  (ROE). 

•  Provide  QA  verification  of  intended  RF  emissions  from  the  ALQ-131.  The  SASY  should  be 
able  to  verify  that  the  electronic  countermeasure  (ECM)  responses  are  in  accordance  with  the 
technique  parameters  programmed  in  the  ALQ-131. 

•  Provide  QA  verification  of  unintended  RF  emissions  in  the  test  environment.  The  SASY 
should  perform  spectrum  surveillance  of  the  test  environment  and  identify  all  unintended  RF 
sources. 

SASY  Site  Alpha  (SSA).  This  is  a  fixed  site.  It  was  used  in  conjunction  with  the  mobile  SASY, 
signal  van  Charlie  (SVC),  to  provide  monitoring  of  intended  RF  emissions  from  threats  and  the 
ECM  responses  from  the  ALQ-131.  The  site’s  antenna  was  slaved  to  the  aircraft  to  capture 
reflections  of  the  ground  threat  emissions  off  the  test  aircraft  and  ECM  responses  from  the  ALQ- 
131.  The  final  utilization  of  SVC  and  SSA  would  be  dependent  on  the  geometry  of  the  profile, 
the  threat  locations,  and  the  site’s  ability  to  accurately  monitor  the  desired  RF  signals.  Because  of 
limitations  at  SVC  described  below,  SSA  was  the  only  site  used  after  the  third  risk  reduction 
mission  to  perform  this  function. 

Signal  Van  Bravo  (SVB).  This  is  a  mobile  SASY  van  which  was  used  to  provide  monitoring  of 
the  RF  environment  to  identify  extraneous  emissions  which  were  not  part  of  the  JADS  test.  It 
uses  an  omnidirectional  antenna,  a  multiband  RF  distribution  system  and  spectrum  analyzers  to 
perform  this  surveillance  function.  This  site  was  not  used  after  the  second  risk  reduction  mission. 
SVB  was  designed  to  monitor  limited  RF  bands  of  interest.  In  this  mode,  SVB  is  acceptable  for 
surveillance  of  nonintentional  RF  emissions.  The  JADS  Electronic  Warfare  (EW)  Test  required 
that  surveillance  not  be  limited  to  a  single  SASY  band.  Revisit  times  to  all  frequencies  of  interest 
became  too  long  for  SVB  to  be  of  use.  The  ALR-69  radar  warning  receiver  (RWR)  on  the  test 
aircraft  was  used  to  perform  the  spectrum  surveillance  function  for  the  test. 

Signal  Van  Charlie  (SVC).  This  is  another  mobile  SASY  van  placed  at  different  locations 
during  risk  reduction  missions  to  determine  the  best  location  for  monitoring  the  JADS  scenario. 
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It  was  used  in  conjunction  with  SSA  to  provide  monitoring  of  intended  RF  emissions  from  the 
threats  and  the  ECM  responses  from  the  ALQ-131.  The  site’s  antenna  was  slaved  to  the  aircraft 
to  capture  reflections  of  the  ground  threat  emissions  from  the  test  aircraft  and  also  ECM 
responses  from  the  ALQ-131.  The  effective  use  of  SVC  is  dependent  on  the  geometry  of  the 
profile,  the  threat  locations,  and  the  sites  ability  to  accurately  monitor  RF  signals.  This  site  was 
not  used  after  the  third  risk  reduction  mission  because  it  was  unable  to  accurately  track  the  test 
aircraft.  The  beam  width  of  the  SVC  antenna  was  too  narrow  to  be  accurately  pointed  at  the  test 
aircraft  using  the  Target  Acquisition  and  Tracking  System  (TATS).  Since  the  signals  of  interest 
could  not  be  collected  without  pointing  the  antenna  at  the  test  aircraft,  the  decision  was  made  to 
delete  SVC  from  the  test  instrumentation  prior  to  the  start  of  open  air  range  (OAR)  baseline 
testing. 

El. 2  Limitations 

The  SVC  signal  van  element  of  SASY  had  a  beam  width  that  was  too  narrow  to  be  accurately 
pointed  at  and  consistently  track  the  aircraft  even  using  the  TATS.  Thus,  the  signals  of  interest 
could  not  be  collected  as  required  by  this  signal  van  and  its  use  was  terminated  early  in  risk 
reduction.  The  pulse  analyzer  was  unable  to  meet  JADS  measurement  requirements  for  jammers 
activation  and  deactivation  times  for  the  following  reasons. 

•  Because  of  collection  limitations  of  the  pulse  analyzer,  a  pulse  buffer  snapshot  approach  was 
used  to  monitor  three  of  the  four  threat  emitters.  The  revisit  time  of  the  pulse  analyzer  site  to 
begin  a  new  snapshot  collection  was  never  less  than  1  second.  In  addition,  if  an  emitter’s 
signal  was  weak,  a  pulse  buffer  collection  would  pause  until  the  signal  strength  increased  to  an 
adequate  level  for  triggering  the  pulse  analyzer  acquisition  hardware.  This  would  require  the 
collection  process  to  be  canceled  prior  to  starting  a  collection  process  for  another  emitter. 
This  cancellation/restart  process  also  increased  the  revisit  time.  Additional  pulse  analyzer’s 
could  have  been  used  to  provide  a  continuous  stare  capability.  However,  because  of  the 
second  limitation  presented  below,  these  additional  units  would  not  have  completely  solved 
the  problem. 

•  Because  of  the  location  of  the  SASY  site  and  the  geometry  of  the  JADS  flight  profile,  RF 
energy  reflected  from  the  aircraft  was  not  always  sufficient  to  provide  a  reliable  source  for 
measurement  by  the  pulse  analyzer.  Unfortunately,  the  regions  of  the  profile  that  caused  the 
most  problems  were  near  the  edges  of  the  threat  engagements  which  were  the  same  regions 
where  sites  and  jammer  responses  were  activated  and  deactivated. 

E2.  Pulse,  Network,  and  Spectrum  Analyzers 

E2.1  Description 

Pulse  Analyzer.  A  SciComm  pulse  analyzer  was  used  to  measure  pulse  parameters  of  threats  and 
ALQ-131  jammer  responses.  This  analyzer  captures  pulse  buffers  and  time  tags  the  time  of  arrival 
for  each  pulse  in  this  buffer.  In  addition,  frequency,  amplitude  and  pulse  width  measurements  are 
made  for  each  pulse  in  the  buffer.  The  time-tag  source  for  the  pulse  analyzer  is  synchronized  to 
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the  range’s  universal  time  code  (UTC).  Since  time-tags  are  available  for  each  pulse  in  the  history 
buffer,  the  pulse  analyzer  can  be  used  to  measure  activation  and  deactivation  times  of  threats  and 
ALQ-131  ECM  responses.  The  pulse  analyzer  does  not  have  the  capability  to  measure 
continuous  wave  (CW)  emitter  parameters.  The  measurement  capabilities  of  the  pulse  analyzer 
are  summarized  below. 

•  Threat  pulse  characteristics 

•  Pulse  repetition  interval  (PRI)  - 100  nanosecond  (ns)  resolution  and  accuracy 

•  Pulse  width  -  100  ns  resolution  and  accuracy 

•  RE  center  frequency  -  5  megahertz  (MHz)  resolution  and  accuracy 

•  Scan  period  -  1  millisecond  (ms)  resolution;  accuracy  of  two  times  the  threat’s  maximum 
PRI 

•  ECM  pulse  characteristics 

•  Technique  wobulation  period  - 1  ms  resolution;  accuracy  of  10%  of  the  true  period 

•  Amplitude  modulation  (AM)  duty  cycle  - 1%  resolution;  10%  accuracy 

•  AM  type 

•  AM  period  -  1  ms  resolution;  accuracy  of  two  times  the  threat’s  maximum  PRI 

•  Measure  activation  times  of  emitters  and  ECM  -  10  ms  resolution  and  accuracy 

•  Measure  deactivation  times  of  emitters  and  ECM  - 100  ms  resolution  and  accuracy 

Network  Analyzer.  The  network  analyzer  provides  a  frequency  versus  time  display  that  is  video 
recorded  for  post-test  analysis.  It  provides  data  about  the  extent  of  the  doppler  shift,  the  shape  of 
the  walk-off  (i.e.,  linear  versus  parabolic),  and  the  number  of  simultaneous  false  targets.  The 
measurement  capabilities  of  the  network  analyzer  are  summarized  below. 

•  Pull-off  frequency  extents  -  100  hertz  (Hz)  resolution  and  accuracy 

•  Pull-off  period  - 10  ms  resolution  and  accuracy 

•  Hold  time  - 10  ms  resolution  and  accuracy 

•  Technique  period  - 10  ms  resolution  and  accuracy 

•  Technique  activation  time  - 10  ms  resolution  and  accuracy 

•  Technique  deactivation  time  -  10  ms  resolution  and  accuracy 

Spectrum  Analyzer.  Spectrum  analyzers  at  the  SASY  sites  provided  a  real-time  qualitative 
assessment  of  jammer  responses  and  threat  emissions.  Their  primary  function  was  to  assess  the 
health  of  test  assets  during  the  mission  to  identify  major  problems  that  may  be  corrected  quickly. 
The  measurement  requirement  for  the  spectrum  analyzers  was  to  provide  visual  displays  of 
emitter  and  ECM  waveforms  for  real-time  verification.  Selected  spectrum  analyzer  displays  were 
recorded  for  post-test  processing. 
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£2.2  Limitations 


The  network  analyzer  was  able  to  provide  the  upper  pull-off  frequency  limit  but  not  at  the 
required  resolution  and  accuracy.  The  available  resolution  and  accuracy  was  1  kilohertz  (kHz). 
The  lower  pull-off  frequency  limit  could  not  be  recovered  because  of  limits  in  resolution  of  the 
display.  The  technique  activation  and  deactivation  times  were  measurable  to  1  second  resolution. 
However,  due  to  the  same  limitations  explained  above  for  the  pulse  analyzer,  these  times  were 
likely  more  inaccurate  than  1  second.  This  inaccuracy  was  due  to  inadequate  signal  levels  from 
the  sites  received  and  analyzed  by  the  SASY  over  the  entire  flight  profile  used  by  JADS. 

The  spectrum  analyzers  were  able  to  provide  real-time  qualitative  verification  of  techniques  for  all 
JADS  threats;  however,  because  of  limited  threat  signal  strength  reflected  from  the  aircraft, 
verification  of  threat  emissions  was  not  possible  over  portions  of  the  flight  path. 

E3.  Threat  Site  Instrumentation 

E3.1  Description 

The  threat  sites  were  instrumented  to  provide  information  on  operator  switch  positions,  threat 
pedestal  position,  and  certain  missile  flyout  parameters.  These  items  are  discussed  below. 

Operator  Switchology.  This  instrumentation  provided  information  about  a  threat’s  operating 
mode  during  a  test  as  well  as  when  the  mode  was  activated  and  deactivated.  Switches  and 
controls  at  the  site  are  monitored  by  instrumentation  which  reports  their  state  at  a  periodic  rate. 
The  switch  states  are  recorded  locally  as  well  as  transmitted,  real  time,  back  to  a  central  recording 
and  display  facility.  These  data  were  then  processed  post-test  to  extract  a  time  history  of  site 
activity  during  the  mission.  The  switchology  instrumentation  was  required  to  measure  significant 
threat  mode  changes  (i.e.,  dummy  load/radiate,  track  mode,  PRI,  frequency,  acquiring  or  tracking 
target,  missile/weapon  activity,  and  state  of  electronic  counter-countermeasures  [ECCM] 
techniques).  In  addition,  these  mode  changes  were  time-tagged  with  UTC.  The  accuracy  of  these 
time-tags  must  be  no  worse  than  the  track  loop/scan  interval  of  the  threat  system  being 
instrumented. 

Pedestal  Position.  Instrumentation  at  each  site  records  the  position  of  the  threat  pedestal  in 
azimuth  and  elevation  as  well  as  the  position  of  angle  and  range  track  gates  where  applicable. 
This  information  is  recorded  for  post-test  processing  to  determine  the  position  of  the  target 
aircraft  as  perceived  by  the  threat.  Differences  between  perceived  and  real  positions  are 
expressed  as  tracking  error.  The  measurement  requirements  for  the  tracked  position 
instrumentation  is  as  follows. 

•  Tracked  position  is  provided  from  each  site  throughout  the  mission  regardless  of  threat  mode 
(including  dummy  load  or  off). 

•  Tracked  angle  position  -  resolution  and  accuracy  of  25%  of  the  threat’s  beam  width. 

•  Tracked  position  is  updated  at  the  same  rate  as  the  switchology  data  described  above. 

•  Tracked  range  position  -  resolution  and  accuracy  better  than  the  threat’s  range  gate  duration. 


88 


Missile  Data.  The  real-time  missile  flyout  models  executed  at  each  surface-to-air  missile  (SAM) 
site  provided  data  for  post-test  determination  of  missile  miss  distance.  Missile  flyout  data  include 
missile  position  versus  time.  These  data  are  then  processed  post-test  with  the  true  aircraft 
position  to  obtain  a  miss  distance  value  for  each  missile  fired.  The  measurement  requirement  for 
this  instrumentation  is  a  single  miss  distance  value  for  each  missile  fired.  The  accuracy  and 
resolution  for  missile  miss  distance  must  not  be  worse  than  twice  the  accuracy  of  the  aircraft 
position  data. 

E3.2  Limitations 

Range  missile  miss  distance  data  were  provided  for  the  Simulated  Air  Defense  System  (SADS)  III 
and  the  SADS  VIII  at  the  requested  resolution  in  a  database  file  format.  No  miss  distance  data 
were  generated  for  the  SADS  VI.  However,  these  data  were  generated  by  the  Air  Force 
Electronic  Warfare  Evaluation  Simulator  (AFEWES)  simulation  of  the  missile.  Miss  distance  data 
were  neither  expected  nor  received  for  the  Weapon  Evaluation  Simulated  Threat  (WEST)  X 
because  of  the  lack  of  a  validated  projectile  flyout  simulation. 

E4.  Radar  Detection  and  Performance  Analysis  System  (RDAPAS) 

E4.1  Description 

The  RDAPAS  instrumentation  collects  log  video,  detected  intermediate  frequency  (IF)  data,  and 
automatic  gain  control  (AGC)  voltages  from  the  SADS  III,  SADS  VIII,  and  WEST  X.  Data 
provided  by  the  RDAPAS  also  include  calibration  data  which  can  be  used  to  derive  the  threats 
absolute  received  power  level  from  the  target.  These  data  may  then  be  used  to  extract  jammer-to- 
signal  ratio  (J/S)  by  taking  advantage  of  the  jammer’s  amplitude  modulation  technique  waveform. 
The  jammer  uses  an  on/off  modulation  which  results  in  received  signals  containing  the  target 
return  only  (S)  and  both  the  target  return  and  jamming  (J+S).  These  values  can  then  be  used  to 
extract  J/S.  The  measurement  requirement  for  the  RDAPAS  is  to  provide  absolute  received 
signal  levels  with  an  accuracy  better  than  3  decibels  (dB).  The  resolution  is  also  required  to  be 
better  than  3  dB.  Measurements  of  received  signal  level  are  required  during  the  entire  threat 
engagement  at  an  update  interval  no  less  than  25  ns. 

E4.2  Limitations 

There  were  limitations  in  the  ability  of  RDAPAS  to  compute  J/S  in  the  range  environment.  These 
limitations  were  due  to  the  instantaneous  dynamic  range  of  the  radar  receiver  and  the  threat 
tracking  performance.  The  measurement  of  the  J/S  was  constrained  by  the  instantaneous  range 
of  the  threat  radar  receiver  at  the  point  where  the  RDAPAS  is  connected  to  the  threat.  If  the  true 
J/S  exceeds  the  instantaneous  dynamic  range  of  the  receiver,  then  the  RDAPAS-derived  J/S  will 
be  smaller  than  the  true  value.  This  result  was  because  either  the  J+S  measurement  interval  would 
drive  the  threat  receiver  into  saturation  (resulting  in  a  smaller  measured  value  for  J+S)  or  the  S 
measurement  interval  would  be  below  the  threat  receiver’s  sensitivity  (resulting  in  a  larger 
measured  value  for  S).  Which  of  these  conditions  occurred  depended  on  the  state  of  the  AGC; 
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however,  either  led  to  smaller  computed  J/S  than  actual.  Also,  during  each  of  these  situations,  the 
actual  J/S  was  relatively  large.  However,  this  inaccurate  information  was  adequate  for  assessing 
the  JADS  test  objectives  related  to  J/S.  The  second  constraint  is  related  to  the  ability  of  the  threat 
to  track  the  target.  The  J/S  computation  was  based  upon  receiving  jamming  and  signal  energy 
into  the  threat  receiver.  However,  if  the  threat  is  not  tracking  the  target  correctly  because  of  large 
angle  track  errors,  then  there  will  be  no  signal  return  or  jammer  response  from  the  aircraft.  But, 
during  these  intervals,  J/S  was  no  longer  defined  and  was  of  no  interest.  Therefore,  the  second 
problem  was  self-curing. 

JADS  was  unable  to  derive  J/S  values  from  the  RDAPAS  data  during  the  range  testing.  This 
inability  was  due  to  tape  formatting  problems,  lack  of  communication  between  the  range  and 
Georgia  Tech  Research  Institute  personnel,  and  RDAPAS  failures  during  the  mission.  The 
RDAPAS  tape  was  provided  on  a  4  millimeter  digital  audio  tape.  The  software  used  by  the  range 
to  generate  this  tape  product  had  several  problems  that  needed  to  be  resolved  before  a  standard 
tape  product  could  be  created.  A  second  problem  with  the  RDAPAS  data  product  was  related  to 
the  reliability  of  the  RDAPAS  systems  at  the  threats.  Three  RDAPAS  systems  were  scheduled 
for  each  risk  reduction  mission.  However,  no  more  than  two  RDAPAS  systems  were  able  to 
provide  data  on  any  single  mission.  As  a  result  of  all  these  problems  and  subsequent  inability  to 
calculate  radar  cross-section,  the  RDAPAS  effort  was  terminated  midway  through  the  open  air 
range  baseline  testing. 

E5.  Ground  Track  Reference  Radar 

E5.1  Description 

The  ground  tracking  reference  radar  provided  a  backup  data  source  to  the  RAJPO  position  data 
and  also  provided  a  target  position  source  used  by  the  SASY  and  threat  systems  for  antenna 
slaving.  Reference  radar  can  be  merged  with  the  RAJPO  data  after  the  mission  using  the  time- 
space-position  information  (TSPI)  Data  Optimizing  Processor  (TDOP).  The  TDOP  is  designed 
to  analyze  multiple  TSPI  data  inputs  and  select  the  best  estimate  of  true  target  aircraft  position. 
The  ground  tracking  radar  also  provides  a  real-time  position  update  of  the  target  which  is  used  as 
an  input  to  the  TATS  which  can  be  transmitted  to  remote  threat  sites.  This  information  is  then 
used  by  sites  to  point  their  antenna  at  the  target  aircraft.  This  is  necessary  to  permit  initial 
acquisition  of  the  target  by  threat  systems  and  to  ensure  that  the  SASY  receives  adequate  signals 
from  the  aircraft  throughout  the  mission.  The  measurement  requirements  for  the  ground  tracking 
radar  are  as  follows. 

•  Real-time  data  -  target  position  data  must  be  accurate  enough  to  point  threat  and  SASY 
antennas  at  the  target  aircraft  during  all  parts  of  the  flight  profile. 

•  Post-test  -  target  position  data  must  have  an  accuracy  and  resolution  no  worse  than  the 
RAJPO  positional  data. 
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E5.2  Limitations 


The  real-time  data  from  the  reference  radar  were  sufficient  to  accurately  point  antennas  for  the 
SADS  III,  SADS  VI,  and  SSA.  However,  pointing  angles  were  not  sufficiently  accurate  to  steer 
the  antennas  for  the  SADS  VIII,  WEST  X,  and  SVC.  The  inability  to  accurately  steer  these  sites 
was  more  likely  the  result  of  latencies  and  update  rates  of  data  transmissions  from  the  reference 
site  to  the  remote  sites  over  the  TATS  system.  The  SADS  VIII  and  WEST  X  had  other 
equipment  available  that  were  used  in  conjunction  with  TATS  to  increase  the  accuracy  of  the 
pointing  angles  provided  by  the  reference  radar.  The  SVC  was  not  used  after  the  second  risk 
reduction  mission  because  of  this  limitation  in  pointing  accuracy.  The  post-test  data  product  did 
not  meet  the  required  measurement  accuracy  and  therefore  was  not  useful  for  JADS  test 
objectives. 
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